Changelog » History » Version 5
Version 4 (Kornelius Kalnbach, 07/04/2011 02:39 AM) → Version 5/9 (Kornelius Kalnbach, 09/14/2011 02:50 AM)
h1=. CodeRay Version History
p=. _This files lists all changes in the CodeRay library since the 0.9.8 0.8.4 release._
{{toc}}
h2. Changes in 1.0
CodeRay 1.0 is a major rewrite of the library, and incompatible to earlier versions.
The command line and programmer interfaces are similar to 0.9, but the internals have completely changed.
h3. General changes
* *NEW*: The new Diff scanner colorizes code inside of the diff, and highlights inline changes.
* *NEW*: Extended support and usage of HTML5 and CSS 3 features.
* *NEW*: Direct Streaming
* *NEW* scanners: Clojure and HAML
* *CHANGED*: Token classes (used as CSS classes) are readable names; breaks you stylesheet!
* *IMPROVED* documentation
* *IMPROVED* speed: faster startup (using @autoload@), scanning, and encoding
* *IMPROVED* Ruby 1.9 encodings support
* *IMPROVED* Tests: There are more of them now!
h3. Direct Streaming
CodeRay 1.0 introduces _Direct Streaming_ as a faster and simpler alternative to Tokens. It means that all Scanners,
Encoders and Filters had to be rewritten, and that older scanners using the Tokens API are no longer compatible with
this version.
The main benefits benefit of this change are:
* is more speed (benchmarks show 10% to 50% more tokens per second compared to CodeRay 0.9)
* the ability to stream output into 0.9), a pipe on the command line
* a simpler API
* API, and less code code.
Changes related to the new tokens handling include:
* *CHANGED*: The Scanners now call Encoders directly; tokens are not added to a Tokens array, but are send to the
Encoder as a method call. The Tokens representation (which can be seen as a cache now) is still present, but as a
special case; Tokens just encodes the given tokens into an Array for later use.
* *CHANGED*: The token actions (@text_token@, @begin_group@ etc.) are now public methods of @Encoder@ and @Tokens@.
* *REWRITE* of all Scanners, Encoders, Filters, and Tokens.
* *RENAMED* @:open@ and @:close@ actions to @:begin_group@ and @:end_group@.
* *RENAMED* @open_token@ and @close_token@ methods to @begin_group@ and @end_group@.
* *NEW* method @#tokens@ allows to add several tokens to the stream. @Tokens@ and @Encoders::Encoder@ define this
method.
* *CHANGED* The above name changes also affect the JSON, XML, and YAML encoders. CodeRay 1.0 output will be incompatible
with earlier versions.
* *REMOVED* @TokenStream@ and the @Streamable@ API and all related features like @NotStreamableError@ are now obsolete
and have been removed.
h3. Command Line General changes
The @coderay@ executable was rewritten * *IMPROVED* documentation in general; additions, corrections and has a few new features:
cleanups.
* *NEW* Ability to stream into a pipe; try @coderay file | more -r@ *NEW*: Extended support and usage of HTML5 and CSS 3 features.
* *NEW* help *IMPROVED* Ruby 1.9 support (_._ not in @$LOAD_PATH@)
* *IMPROVED* speed of HTML encoding when using CSS classes.
* *IMPROVED*: more consistent parameter handling Faster startup by replacing @require@ with @autoload@. CodeRay
features will now only be loaded when they're needed.
* *REMOVED* @coderay_stylesheet@ executable; use @coderay stylesheet [name]@. *FIXED* some image links in the documentation.
* *ADDED* a lot of tests.
h3. @Tokens@ Helpers
The helper classes were cleaned up; see below for details.
* *NEW* methods @encode_with@, @count@, @begin_group@, @end_group@, @begin_line@, *CHANGED* @Plugin@ API was simplified and @end_line@. stripped of all unnecessary features.
* *CHANGED* Moved @GZip@ and @FileType@ libraries into @CodeRay@; cleaned them up.
h3. @Tokens@
* *REMOVED* methods @#stream?@, @#each_text_token@.
* *REMOVED* @#text@ and @#text_size@ methods. Use the @Text@ encoder instead.
* *REMOVED* special implementation of @#each@ taking a filter parameter. Use @TokenKindFilter@ instead.
* *NEW* methods @encode_with@, @count@, @begin_group@, @end_group@, @begin_line@, and @end_line@.
h3. *RENAMED*: @TokenKinds@
Renamed from @Tokens::ClassOfKind@ (was also @Tokens::AbbreviationForKind@ for a while).
The term "token class" is no longer used in CodeRay. Instead, tokens have _kinds_.
See "#122":http://redmine.rubychan.de/issues/122.
* *CHANGED* all token CSS classes to readable names.
* *ADDED* token kinds @:filename@, @:namespace@, and @:eyecatcher@.
* *RENAMED* @:pre_constant@ and @:pre_type@ to @:predefined_constant@ and @predefined_type@.
* *RENAMED* @:oct@ and @:bin@ to @:octal@ and @binary@.
* *REMOVED* token kinds @:attribute_name_fat@, @:attribute_value_fat@, @:operator_fat@, @interpreted@,
@:tag_fat@, @tag_special@, @:xml_text@, @:nesting_delimiter@, @:open@, and @:close@.
* *ADDED* token kinds @:filename@, @:namespace@, and @:eyecatcher@.
* *CHANGED*: Don't raise error for unknown token kinds unless in @$CODERAY_DEBUG@ mode.
* *CHANGED* the value for a token kind that is not highlighted from
@:NO_HIGHLIGHT@ to @false@.
h3. @Duo@
* *NEW* method @call@ for allowing code like @CodeRay::Duo[:python => :yaml].(code)@ in Ruby 1.9.
h3. @Encoders::CommentFilter@
* *NEW* alias @:remove_comments@
h3. @Encoders::Filter@
* *NEW* option @tokens@.
* *CHANGED*: Now it simply delegates to the output.
* *REMOVED* @include_text_token?@ and @include_block_token?@ methods.
h3. @Encoders::HTML@
The HTML encoder was cleaned up and simplified.
* *CHANGED* the default style to @:alpha@.
* *NEW*: HTML5 and CSS 3 compatible.
See "#215":http://redmine.rubychan.de/issues/215.
* *ADDED* support for @:line_number_anchors@.
See "#208":http://redmine.rubychan.de/issues/208.
* *CHANGED* the default style to @:alpha@.
* *CHANGED*: Use double click to toggle line numbers in table mode (as single
click jumps to an anchor.)
* *REMOVED* support for @:line_numbers => :list@.
* *FIXED* splitting of lines for @:line_numbers => :inline@, so that the line
numbers don't get colored, too.
* *RENAMED* @Output#numerize@ to @#number@, which is an actual English word.
h3. @Encoders::LinesOfCode@
* *CHANGED*: @compile@ and @finish@ methods are now protected.
h3. *Renamed*: @Encoders::Terminal@ (was @Encoders::Term@)
* *RENAMED* from @Encoders::Term@, added @:term@ alias.
* *CLEANUP*: Use @#setup@'s @super@, don't use @:procedure@ token class.
* *CHANGED*: @#token@'s second parameter is no longer optional.
* *REMOVED* colors for obsolete token kinds.
* *FIXED* handling of line tokens.
h3. @Encoders::Text@
* *FIXED* default behavior of stripping the trailing newline.
h3. *RENAMED*: @Encoders::TokenKindFilter@ (was @Encoders::TokenClassFilter@)
Renamed from @TokenClassFilter@.
* *NEW*: Handles token groups.
See "#223":http://redmine.rubychan.de/issues/223.
* *RENAMED* @include_block_token?@ to @include_group?@.
h3. @Encoders::Statistic@
* *CHANGED*: Tokens actions are counted separately.
h3. @Scanners::Scanner@
* *REMOVED* helper method @String#to_unix@.
* *REMOVED* method @#streamable?@.
* *REMOVED* @#marshal_load@ and @#marshal_dump@.
* *RENAMED* class method @normify@ to @normalize@; it also deals with encoding now.
* *NEW* methods @#file_extension@ and @#encoding@.
* *NEW*: The @#tokenize@ method also takes an Array of Strings as source. The
code is highlighted as one and split into parts of the input lengths
after that using @Tokens#split_into_parts@.
* *NEW* method @#binary_string@
* *REMOVED* helper method @String#to_unix@.
* *REMOVED* method @#streamable?@.
* *REMOVED* @#marshal_load@ and @#marshal_dump@.
* *RENAMED* class method @normify@ to @normalize@; it also deals with encoding now.
* *CHANGED*: @#column@ starts counting with 1 instead of 0
h3. *NEW*: @Scanners::Clojure@
Thanks to Licenser, CodeRay now supports the Clojure language.
h3. @Scanners::CSS@
* *NEW*: Rudimentary support for the @attr@, @counter@, and @counters@ functions.
See "#224":http://redmine.rubychan.de/issues/224.
* *NEW*: Rudimentary support for CSS 3 colors.
* *CHANGED*: Attribute selectors are highlighted as @:attribute_name@ instead of @:string@.
* *CHANGED*: Comments are scanned as one token instead of three.
h3. @Scanners::Debug@
* *NEW*: Support for line tokens (@begin_line@ and @end_line@ represented by @[@ and @]@.)
* *FIXED*: Don't send @:error@ and @nil@ tokens for buggy input any more.
* *FIXED*: Closes unclosed tokens at the end of @scan_tokens@.
* *IMPROVED*: Highlight unknown tokens as @:error@.
* *CHANGED*: Raises an error when trying to end an invalid token group.
h3. @Scanners::Delphi@
* *FIXED*: Closes open string groups.
h3. @Scanners::Diff@
* *NEW*: Highlighting of code based on file names.
See ticket "#52":http://redmine.rubychan.de/issues/52.
Use the @:highlight_code@ option to turn this feature off. It's enabled
by default.
This is a very original feature. It enables multi-language highlighting for
diff files, which is especially helpful for CodeRay development itself. The
updated version of the scanner test suite generated .debug.diff.html files
using this.
Note: This is still experimental. Tokens spanning more than one line
may get highlighted incorrectly. CodeRay tries to keep scanner states
between the lines and changes, but the quality of the results depend on
the scanner.
* *NEW*: Inline change highlighting, as suggested by Eric Thomas.
See ticket "#227":http://redmine.rubychan.de/issues/227 for details.
Use the @:inline_diff@ option to turn this feature off. It's enabled by
default.
For single-line changes (that is, a single deleted line followed by a single
inserted line), this feature surrounds the changed parts with an
@:eyecatcher@ group which appears in a more saturated background color.
The implementation is quite complex, and highly experimental. The problem
with multi-layer tokenizing is that the tokens have to be split into parts.
If the inline change starts, say, in the middle of a string, then additional
@:end_group@ and @:begin_group@ tokens must be inserted to keep the group
nesting intact. The extended @Scanner#tokenize@ method and the new
@Tokens#split_into_parts@ method take care of this.
* *NEW*: Highlight the file name in the change headers as @:filename@.
* *CHANGED*: Highlight unknown lines as @:comment@ instead of @:head@.
* *IMPROVED*: Background colors for Diff output have been optimized.
h3. *RENAMED*: @Scanners::ERB@ (was @Scanners::RHTML@)
h3. *NEW*: @Scanners::HAML@
It uses the new :state options of the HTML and Ruby scanners.
Some rare cases are not considered (like @#{...}@ snippets inside of :javascript blocks),
but it highlights pretty well.
h3. @Scanners::HTML@
* *FIXED*: Closes open string groups.
h3. @Scanners::JavaScript@
* *IMPROVED*: Added @NaN@ and @Infinity@ to list of predefined constants.
* *IMPROVED* recognition of RegExp literals with leading spaces.
h3. @Scanners::Java@
* *NEW*: Package names are highlighted as @:namespace@.
See "#210":http://redmine.rubychan.de/issues/210.
h3. *REMOVED*: @Scanners::NitroXHTML@
Nitro is "dead":http://www.nitrohq.com/.
h3. *RENAMED*: @Scanners::Text@ (was @Scanners::Plaintext@)
* *IMPROVED*: Just returns the string without scanning (faster).
This is much faster than scanning until @/\z/@ in Ruby 1.8.
h3. @Scanners::Python@
* *CHANGED*: Docstrings are highlighted as @:comment@.
See "#190":http://redmine.rubychan.de/issues/190.
h3. *NEW*: @Scanners::Raydebug@
Copied from @Scanners::Debug@, highlights the token dump instead of importing it. It also reacts to the @.raydebug@ file
name suffix now.
h3. @Scanners::Ruby@
* *ADDED* more predefined keywords (see http://murfy.de/ruby-constants).
* *IMPROVED* support for singleton method definitions.
See "#147":http://redmine.rubychan.de/issues/147.
* *FIXED*: Don't highlight methods with a capital letter as constants
(eg. @GL.PushMatrix@). GL.PushMatrix).
* *NEW*: Highlight buggy floats (like .5) as @:error@.
* *CLEANUP* of documentation, names of constants and variables, state handling.
Moved @StringState@ class from @patterns.rb@ into a separate file.
* *NEW*: Complicated rule for recognition of @foo=@ style method names.
* *NEW*: Handles @:keep_state@ option (a bit; experimental).
Actually, Ruby checks if there is @[~>=]@, but not @=>@ following the name.
* *REMOVED* @EncodingError@
h3. *REMOVED* @Scanners::Scheme@
* It is too buggy, and nobody was using it. To be added again when it's fixed.
See "#59":http://redmine.rubychan.de/issues/59. *CHANGED*: Does use @:operator@ instead of @:operator_fat@ now.
h3. @Scanners::SQL@
* *IMPROVED*: Extended list of keywords and functions (thanks to
Joshua Galvez, Etienne Massip, and others).
See "#221":http://redmine.rubychan.de/issues/221.
* *FIXED*: Closes open string groups.
* *FIXED*: Words after @.@ are always recognized as @:ident@.
h3. @Scanners::YAML@
* *FIXED*: Allow spaces before colon in mappings.
See "#231":http://redmine.rubychan.de/issues/231.
h3. *NEW*: @Styles::Alpha@
A style that uses transparent HSLA colors as defined in CSS 3.
See "#199":http://redmine.rubychan.de/issues/199.
It also uses the CSS 3 property @user-select: none@ to keep the user from selecting the line numbers. This is especially
nice for @:inline@ line numbers.
See "#226":http://redmine.rubychan.de/issues/226.
h3. @WordList@
Stripped down to 19 LOC.
* *RENAMED* @CaseIgnoringWordList@ to @WordList::CaseIgnoring@.
* *REMOVED* caching option because it creates memory leaks.
* *REMOVED* block option.
h3. @FileType@
* *REMOVED* @FileType#shebang@ is a protected method now.
* *NEW*: Recognizes @.gemspec@, @.rjs@, @.rpdf@ extensions, @Gemfile@, and @Capfile@ as Ruby.
Thanks to the authors of the TextMate Ruby bundle!
* *REMOVED* @FileType#shebang@ is a protected method now.
h3. @Plugin@
* *IMPROVED*: @register_for@ sets the @plugin_id@; it can now be a @Symbol@.
* *ADDED* @PluginHost#const_missing@ method: Plugins are loaded automatically.
Using @Scanners::JavaScript@ in your code loads @scanners/java_script.rb@.
* *ADDED* @#all_plugins@ method to simplify getting
information about all available plugins (suggested by bnhymn).
* *CHANGED* the default plugin key from @nil@ to @:default@.
h3. @GZip@
* *MOVED* into @CodeRay@ namespace.
* *MOVED* file from @gzip_simple.rb@ to @gzip.rb@.
* *REMOVED* @String@ extensions.
h3. More Internal API changes
* *FIXED* @Encoders::HTML#token@'s second parameter is no longer optional.
* *CHANGED* @Encoders::HTML::Output@'s API.
* *REMOVED* lots of unused methods.
h2. Changes in 0.9.8 "banister" [2011-05-01]
Fixes for JRuby's 1.9 mode and minor issues.
h3. Rake tasks
* *REMOVED* obsolete @has_rdoc@ gem specification, fixing a warning.
h3. @Scanners::Scanner@
* *NEW* method @#scan_rest@ replaces @scan_until(/\z/)@, which is broken in JRuby 1.6 --1.9 mode.
See "#297":http://redmine.rubychan.de/issues/297.
h3. @Scanners::CSS@
* *FIXED* LOC counting (should be 0).
See "#296":http://redmine.rubychan.de/issues/296.
h3. @Scanners::Ruby@
* *FIXED* the @IDENT@ pattern not to use character properties, which are broken in JRuby 1.6 --1.9 mode.
See "#297":http://redmine.rubychan.de/issues/297, thanks to banister for reporting!
h3. @Scanners::SQL@
* *ADDED* more keywords: @between@, @databases@, @distinct@, @fields@, @full@, @having@, @is@, @prompt@, @tables@.
See "#221":http://redmine.rubychan.de/issues/221, thanks to Etienne Massip again.
h3. @FileType@
* *NEW* regonizes ColdFusion file type extensions @.cfm@ and @.cfc@ as XML.
See "#298":http://redmine.rubychan.de/issues/298, thanks to Emidio Stani.
h2. Changes in 0.9.7 "Etienne" [2011-01-14]
Fixes a dangerous JavaScript scanner bug, and a testing problem with Ruby 1.9.1.
h3. Tests
* *FIXED* The helper classes were cleaned up; see above functional tests now load the lib directory (instead of the gem) in Ruby 1.9.1.
h3. @Scanners::JavaScript@
* *FIXED* @KEY_CHECK_PATTERN@ regexp
See "#264":http://redmine.rubychan.de/issues/264, thanks to Etienne Massip!
h2. Changes in 0.9.6 "WoNáDo" [2010-11-25]
Minor improvements to the Ruby scanner and a fix for details. Ruby 1.9.
h3. @Scanners::Ruby@
* *IMPROVED* handling of new hash syntax (keys are marked as @:key@ now,
colon is a separate @:operator@ token, all idents can be used as keys)
See "#257":http://code.licenser.net/issues/257, thanks to WoNáDo!
* *ADDED* @__ENCODING__@ magic constant (Ruby 1.9)
* *FIXED*: Scanner no longer tries to modify the input string on Ruby 1.9.
See "#260":http://code.licenser.net/issues/260, thanks to Jan Lelis!
h2. Changes in 0.9.5 "Germany.rb" [2010-09-28]
Support for Rubinius ("#251":http://redmine.rubychan.de/issues/251), improved mutlibyte handling, Ruby 1.9 syntax, and valid HTML.
h3. @Encoders::HTML@
* *FIXED*: Line tokens use @span@ with @display: block@ instead of @div@, which was invalid HTML ("#255":http://redmine.rubychan.de/issues/255).
h3. @Scanner::Scanner@
* *IMPROVED* handling of encodings in Ruby 1.9: UTF-8 and Windows-1252 are checked.
* *NEW*: Invalid chars will be converted to @?@ in Ruby 1.9.
* *FIXED* @string=@ method for Rubinius. See "issue 481":http://github.com/evanphx/rubinius/issues/481 on their site.
h3. @Scanners::CSS@
* *FIXED*: Don't use non-ASCII regexps.
h3. @Scanners::Diff@
* *FIXED*: Highlight unexpected lines as @:comment@.
h3. @Scanners::PHP@
* *FIXED*: Use @ASCII-8BIT@ encoding for now.
h3. @Scanners::Ruby@
* *ADDED* support for some Ruby 1.9 syntax ("#254":http://redmine.rubychan.de/issues/254):
** the @->@ lambda shortcut
** new Hash syntax using colons (@{ a: b }@)
* *FIXED*: Use @UTF-8@ encoding.
* *IMPROVED* unicode support on Ruby 1.8 ("#253":http://redmine.rubychan.de/issues/253).
* *FIXED* recognition of non-ASCII identifiers in Ruby 1.9, JRuby, and Rubinius ("#253":http://redmine.rubychan.de/issues/253).
* *CHANGED* @Plugin@ API heredoc recognition to ignore delimiters starting with a digit. This is incorrect, but causes less false positives.
h3. @Scanners::SQL@
* *FIXED* scanning of comments; nice catch, Rubinius!
("#252":http://redmine.rubychan.de/issues/252)
h2. Changes in 0.9.4 "Ramadan" [2010-08-31]
Updated command line interface and minor scanner fixes for the Diff, HTML, and RHTML scanners.
h3. @coderay@ executable
* *FIXED*: Partly rewritten, simplified, fixed.
("#244":http://redmine.rubychan.de/issues/244)
h3. @Scanners::Diff@
* *FIXED* handling of change headers with code on the same line as the @@ marker.
("#247":http://redmine.rubychan.de/issues/242)
h3. @Scanners::HTML@
* *FIXED* a missing regexp modifier that slowed down the scanning.
("#245":http://redmine.rubychan.de/issues/245)
h3. @Scanners::RHTML@
* *FIXED* highlighting of ERB comment blocks.
("#246":http://redmine.rubychan.de/issues/246)
h2. Changes in 0.9.3 "Eyjafjallajökull" [2010-04-18]
* *FIXED*: Documentation of Tokens.
("#218":http://redmine.rubychan.de/issues/218)
h3. @coderay@ executable
* *NEW*: automatic TTY detection (uses @Term@ encoder)
* *NEW*: optional 3rd parameter for the filename
* *FIXED*: Converted to UNIX format.
* *FIXED*: Warn about generated files.
* *FIXED*: Ensure line break after the output (especially for LoC counter).
h3. @Scanners::JavaScript@
* *FIXED*: Don't keep state of XML scanner between calls for E4X literals.
h3. @Scanners::Java@, @Scanners::JSON@
* *FIXED*: Close unfinished strings with the correct token kind.
h2. Changes in 0.9.2 "Flameeyes" [2010-03-14]
* *NEW* Basic tests and a _Rakefile_ are now included in the Gem. [Flameeyes]
A @doc@ task is also included.
* *FIXED* Use @$CODERAY_DEBUG@ for debugging instead of @$DEBUG@. [Trans]
("#192":http://redmine.rubychan.de/issues/192)
* *REMOVED* @Term::Ansicolor@ was simplified bundled under _lib/_, but not used. [Flameeyes]
("#205":http://redmine.rubychan.de/issues/205)
* *WORKAROUND* for Ruby bug
"#2745":http://redmine.ruby-lang.org/issues/show/2745
h3. @Encoders::Term@
* *FIXED* strings are closed correctly
("#138":http://redmine.rubychan.de/issues/138)
* *FIXED* several token kinds had no associated color
("#139":http://redmine.rubychan.de/issues/139)
* *NEW* alias @terminal@
*NOTE:* This encoder will be renamed to @Encoders::Terminal@ in the next release.
h3. @Scanners::Debug@
* *FIXED* Don't close tokens that are not open. Send @:error@ token instead.
h3. @Scanners::Groovy@
* *FIXED* token kind of closing brackets is @:operator@ instead of @nil@
("#148":http://redmine.rubychan.de/issues/148)
h3. @Scanners::PHP@
* *FIXED* allow @\@ operator (namespace separator)
("#209":http://redmine.rubychan.de/issues/209)
h3. @Scanners::YAML@
* *FIXED* doesn't send debug tokens when @$DEBUG@ is true [Trans]
("#149":http://redmine.rubychan.de/issues/149)
h2. Changes in 0.9.1 [2009-12-31]
h3. Token classes
* *NEW* token classes @:complex@, @:decorator@, @:imaginary@
(all for Python)
* *REMOVED* token class @:procedure@
– use @:function@ or @:method@ instead.
h3. @Tokens@
* *NEW* method @#scanner@
Stores the scanner.
* *REMOVED* methods @.write_token@, @.read_token@, @.escape@, @.unescape@
They were only used by the @Tokens@ encoder, which was removed also.
h3. @Encoders::Encoder@
* *REMOVED* Don't require the _stringio_ library.
* *NEW* public methods @#open_token@, @#close_token@, @#begin_line@, @#end_line@
These methods are called automatically, like @#text_token@.
* *NEW* proteced method @#append_encoded_token_to_output@
h3. @Encoders::Tokens@
* *REMOVED* – use @Tokens#dump@ and stripped @Tokens.load@.
h3. @Encoders::Filter@
* *NEW*
A @Filter@ encoder has another @Tokens@ instance as output.
h3. @Encoders::TokenClassFilter@
* *NEW*
It takes 2 options, @:exclude@ and @:include@, that specify which token classes
to include or exclude for the output. They can be a single token class,
an @Array@ of classes, or the value @:all@.
h3. @Encoders::CommentFilter@
* *NEW*
Removes tokens of the @:comment@ class.
h3. @Encoders::LinesOfCode@
* *NEW*
Counts the lines of code according to the @KINDS_NOT_LOC@ token class list
defined by the scanner. It uses the new @TokenClassFilter@.
Alias: @:loc@, as in @tokens.loc@.
h3. @Encoders::JSON@
* *NEW*
Outputs tokens in a simple JSON format.
h3. @Encoders::Term@
* *NEW* (beta, by Rob Aldred)
Outputs code highlighted for a color terminal.
h3. @Encoders::HTML@
* *NEW* option @:title@ (default value is _CodeRay output_)
Setting this changes the title of the HTML page.
* *NEW* option @:highlight_lines@ (default: @nil@)
Highlights the given set of line numbers.
- *REMOVED* option @:level@
It didn't do anything. CodeRay always outputs XHTML.
h3. @Encoders::Text@
* Uses @Encoder@ interface with @super@ and @#text_token@.
h3. @Encoders::XML@
* @FIXED@ ("#94":http://redmine.rubychan.de/issues/94)
It didn't work at all.
h3. Scanners
* *NEW* Mapped @:h@ to @:c@, @:cplusplus@ and @:'c++'@ to @:cpp@,
@:ecma@, @:ecmascript@, @:ecma_script@ to @:java_script@,
@:pascal@ to @:delphi@, and @:plain@ to @:plaintext@.
h3. @Scanners::Scanner@
* *NEW* constant @KINDS_NOT_LOC@
A list of all unnecessary features. token classes not considered in LOC count.
Added appropriate values for scanners.
* *CHANGED* Moved @GZip@ *NEW* method @#lang@ returns the scanner's lang, which is its @plugin_id@.
* *FIXED* automatic, safe UTF-8 detection _[Ruby 1.9]_
* *FIXED* column takes care of multibyte encodings _[Ruby 1.9]_
* *FIXED* is dumpable (@Tokens@ store their scanner in an @@scanner@ variable)
h3. @Scanners::Cpp@
* *NEW* (C++)
h3. @Scanners::Groovy@
* *NEW* (beta)
h3. @Scanners::Python@
* *NEW*
h3. @Scanners::PHP@
* *NEW* (based on Stefan Walk's work)
h3. @Scanners::SQL@
* *NEW* (based on code by Josh Goebel)
h3. @Scanners::C@
* *IMPROVED* added a list of @:directive@ tokens that were @:reserved@ before
* *IMPROVED* detection of labels
* *IMPROVED* allow @1L@ and @1LL@ style literals
h3. @Scanners::CSS@
* *IMPROVED* element selectors are highlighted as @:type@ instead of @:keyword@
h3. @Scanners::Delphi@
* *IMPROVED* Don't cache tokens in CaseIgnoringWordList.
h3. @Scanners::Java@
* *IMPROVED* @assert@ is highlighted as a @:keyword@ now
* *IMPROVED* @const@ and @goto@ are highlighted as @:reserved@
* *IMPROVED* @false@, @true@, and @null@ are highlighted as @:pre_constant@
* *IMPROVED* @threadsafe@ is no longer a @:directive@
* *IMPROVED* @String@ is highlighted as a @:pre_type@
* *IMPROVED* built-in classes ending with _Error_ or _Exception_ are
highlighted as a @:exception@ instead of @:pre_type@
h3. @Scanners::JavaScript@
* *NEW* a list of @PREDEFINED_CONSTANTS@ to be highlighted as @:pre_constant@
* *NEW* XML literals are recognized and highlighted
* *NEW* function name highlighting
* *IMPROVED* @.1@ is highlighted a number
* *FIXED* strings close with the correct kind when terminated unexpectedly
h3. @Scanners::JSON@
* *IMPROVED* constants (@true@, @false@, @nil@) are highlighted as @:value@
h3. @Scanners::Ruby@
* *IMPROVED* @Patterns::KEYWORDS_EXPECTING_VALUE@ for more accurate
@value_expected@ detection
* *IMPROVED* handling of @\@ as a string delimiter
* *IMPROVED* handling of unicode strings; automatic switching to unicode
* *IMPROVED* highlighting of @self.method@ definitions
* *REMOVED* @Patterns::FANCY_START_SAVE@ (obsolete)
* *FIXED* encoding issues _[Ruby 1.9]_
* *FIXED* a problem in early Ruby 1.8.6 patch versions with @Regexp.escape@
h3. @Scanners::YAML@
* *IMPROVED* indentation detection
h3. @Styles::Cycnus@
* changed a few colors (exceptions, inline strings, predefined types)
h3. @Plugin@
* *NEW* method @#title@
Set and get the plugin's title. Titles can be arbitrary strings.
* *NEW* method @#helper@ loads helpers from different plugins
Use this syntax: @helper 'other_plugin/helper_name'@
h3. @FileType@ libraries into @CodeRay@; cleaned them up.
* *NEW* @FileType[]@ takes @Pathname@ instances
* *NEW* regonizes @.cc@, @.cpp@, @.cp@, @.cxx@, @.c++@, @.C@, @.hh@, @.hpp@, @.h++@, @.cu@ extensions (C++)
Thanks to Sander Cox and the TextMate C bundle.
* *NEW* regonizes @.pas@, @.dpr@ extensions (Delphi)
* *NEW* regonizes @.gvy@, @.groovy@ extensions (Groovy)
* *NEW* regonizes @.php@, @.php3@, @.php4@, @.php5@ extensions (PHP)
* *NEW* regonizes @.py@, @.py3@, @.pyw@ extensions (Python)
* *NEW* regonizes @.rxml@ extension (Ruby)
* *NEW* regonizes @.sql@ extension (SQL)
* File types list was sorted alphabetically.
h3. @CaseIgnoringWordList@
* *FIXED* ("#97":http://redmine.rubychan.de/issues/97)
The default value is no longer ignored.
h3. @ForRedCloth@
* *FIXED* for RedCloth versions 4.2.0+ ("#119":http://redmine.rubychan.de/issues/119)
h3. Cleanups
* warnings about character classes _[Ruby 1.9]_
* encoding issues _[Ruby 1.9]_
* documentation, code
p=. _This files lists all changes in the CodeRay library since the 0.9.8 0.8.4 release._
{{toc}}
h2. Changes in 1.0
CodeRay 1.0 is a major rewrite of the library, and incompatible to earlier versions.
The command line and programmer interfaces are similar to 0.9, but the internals have completely changed.
h3. General changes
* *NEW*: The new Diff scanner colorizes code inside of the diff, and highlights inline changes.
* *NEW*: Extended support and usage of HTML5 and CSS 3 features.
* *NEW*: Direct Streaming
* *NEW* scanners: Clojure and HAML
* *CHANGED*: Token classes (used as CSS classes) are readable names; breaks you stylesheet!
* *IMPROVED* documentation
* *IMPROVED* speed: faster startup (using @autoload@), scanning, and encoding
* *IMPROVED* Ruby 1.9 encodings support
* *IMPROVED* Tests: There are more of them now!
h3. Direct Streaming
CodeRay 1.0 introduces _Direct Streaming_ as a faster and simpler alternative to Tokens. It means that all Scanners,
Encoders and Filters had to be rewritten, and that older scanners using the Tokens API are no longer compatible with
this version.
The main benefits benefit of this change are:
* is more speed (benchmarks show 10% to 50% more tokens per second compared to CodeRay 0.9)
* the ability to stream output into 0.9), a pipe on the command line
* a simpler API
* API, and less code code.
Changes related to the new tokens handling include:
* *CHANGED*: The Scanners now call Encoders directly; tokens are not added to a Tokens array, but are send to the
Encoder as a method call. The Tokens representation (which can be seen as a cache now) is still present, but as a
special case; Tokens just encodes the given tokens into an Array for later use.
* *CHANGED*: The token actions (@text_token@, @begin_group@ etc.) are now public methods of @Encoder@ and @Tokens@.
* *REWRITE* of all Scanners, Encoders, Filters, and Tokens.
* *RENAMED* @:open@ and @:close@ actions to @:begin_group@ and @:end_group@.
* *RENAMED* @open_token@ and @close_token@ methods to @begin_group@ and @end_group@.
* *NEW* method @#tokens@ allows to add several tokens to the stream. @Tokens@ and @Encoders::Encoder@ define this
method.
* *CHANGED* The above name changes also affect the JSON, XML, and YAML encoders. CodeRay 1.0 output will be incompatible
with earlier versions.
* *REMOVED* @TokenStream@ and the @Streamable@ API and all related features like @NotStreamableError@ are now obsolete
and have been removed.
h3. Command Line General changes
The @coderay@ executable was rewritten * *IMPROVED* documentation in general; additions, corrections and has a few new features:
cleanups.
* *NEW* Ability to stream into a pipe; try @coderay file | more -r@ *NEW*: Extended support and usage of HTML5 and CSS 3 features.
* *NEW* help *IMPROVED* Ruby 1.9 support (_._ not in @$LOAD_PATH@)
* *IMPROVED* speed of HTML encoding when using CSS classes.
* *IMPROVED*: more consistent parameter handling Faster startup by replacing @require@ with @autoload@. CodeRay
features will now only be loaded when they're needed.
* *REMOVED* @coderay_stylesheet@ executable; use @coderay stylesheet [name]@. *FIXED* some image links in the documentation.
* *ADDED* a lot of tests.
h3. @Tokens@ Helpers
The helper classes were cleaned up; see below for details.
* *NEW* methods @encode_with@, @count@, @begin_group@, @end_group@, @begin_line@, *CHANGED* @Plugin@ API was simplified and @end_line@. stripped of all unnecessary features.
* *CHANGED* Moved @GZip@ and @FileType@ libraries into @CodeRay@; cleaned them up.
h3. @Tokens@
* *REMOVED* methods @#stream?@, @#each_text_token@.
* *REMOVED* @#text@ and @#text_size@ methods. Use the @Text@ encoder instead.
* *REMOVED* special implementation of @#each@ taking a filter parameter. Use @TokenKindFilter@ instead.
* *NEW* methods @encode_with@, @count@, @begin_group@, @end_group@, @begin_line@, and @end_line@.
h3. *RENAMED*: @TokenKinds@
Renamed from @Tokens::ClassOfKind@ (was also @Tokens::AbbreviationForKind@ for a while).
The term "token class" is no longer used in CodeRay. Instead, tokens have _kinds_.
See "#122":http://redmine.rubychan.de/issues/122.
* *CHANGED* all token CSS classes to readable names.
* *ADDED* token kinds @:filename@, @:namespace@, and @:eyecatcher@.
* *RENAMED* @:pre_constant@ and @:pre_type@ to @:predefined_constant@ and @predefined_type@.
* *RENAMED* @:oct@ and @:bin@ to @:octal@ and @binary@.
* *REMOVED* token kinds @:attribute_name_fat@, @:attribute_value_fat@, @:operator_fat@, @interpreted@,
@:tag_fat@, @tag_special@, @:xml_text@, @:nesting_delimiter@, @:open@, and @:close@.
* *ADDED* token kinds @:filename@, @:namespace@, and @:eyecatcher@.
* *CHANGED*: Don't raise error for unknown token kinds unless in @$CODERAY_DEBUG@ mode.
* *CHANGED* the value for a token kind that is not highlighted from
@:NO_HIGHLIGHT@ to @false@.
h3. @Duo@
* *NEW* method @call@ for allowing code like @CodeRay::Duo[:python => :yaml].(code)@ in Ruby 1.9.
h3. @Encoders::CommentFilter@
* *NEW* alias @:remove_comments@
h3. @Encoders::Filter@
* *NEW* option @tokens@.
* *CHANGED*: Now it simply delegates to the output.
* *REMOVED* @include_text_token?@ and @include_block_token?@ methods.
h3. @Encoders::HTML@
The HTML encoder was cleaned up and simplified.
* *CHANGED* the default style to @:alpha@.
* *NEW*: HTML5 and CSS 3 compatible.
See "#215":http://redmine.rubychan.de/issues/215.
* *ADDED* support for @:line_number_anchors@.
See "#208":http://redmine.rubychan.de/issues/208.
* *CHANGED* the default style to @:alpha@.
* *CHANGED*: Use double click to toggle line numbers in table mode (as single
click jumps to an anchor.)
* *REMOVED* support for @:line_numbers => :list@.
* *FIXED* splitting of lines for @:line_numbers => :inline@, so that the line
numbers don't get colored, too.
* *RENAMED* @Output#numerize@ to @#number@, which is an actual English word.
h3. @Encoders::LinesOfCode@
* *CHANGED*: @compile@ and @finish@ methods are now protected.
h3. *Renamed*: @Encoders::Terminal@ (was @Encoders::Term@)
* *RENAMED* from @Encoders::Term@, added @:term@ alias.
* *CLEANUP*: Use @#setup@'s @super@, don't use @:procedure@ token class.
* *CHANGED*: @#token@'s second parameter is no longer optional.
* *REMOVED* colors for obsolete token kinds.
* *FIXED* handling of line tokens.
h3. @Encoders::Text@
* *FIXED* default behavior of stripping the trailing newline.
h3. *RENAMED*: @Encoders::TokenKindFilter@ (was @Encoders::TokenClassFilter@)
Renamed from @TokenClassFilter@.
* *NEW*: Handles token groups.
See "#223":http://redmine.rubychan.de/issues/223.
* *RENAMED* @include_block_token?@ to @include_group?@.
h3. @Encoders::Statistic@
* *CHANGED*: Tokens actions are counted separately.
h3. @Scanners::Scanner@
* *REMOVED* helper method @String#to_unix@.
* *REMOVED* method @#streamable?@.
* *REMOVED* @#marshal_load@ and @#marshal_dump@.
* *RENAMED* class method @normify@ to @normalize@; it also deals with encoding now.
* *NEW* methods @#file_extension@ and @#encoding@.
* *NEW*: The @#tokenize@ method also takes an Array of Strings as source. The
code is highlighted as one and split into parts of the input lengths
after that using @Tokens#split_into_parts@.
* *NEW* method @#binary_string@
* *REMOVED* helper method @String#to_unix@.
* *REMOVED* method @#streamable?@.
* *REMOVED* @#marshal_load@ and @#marshal_dump@.
* *RENAMED* class method @normify@ to @normalize@; it also deals with encoding now.
* *CHANGED*: @#column@ starts counting with 1 instead of 0
h3. *NEW*: @Scanners::Clojure@
Thanks to Licenser, CodeRay now supports the Clojure language.
h3. @Scanners::CSS@
* *NEW*: Rudimentary support for the @attr@, @counter@, and @counters@ functions.
See "#224":http://redmine.rubychan.de/issues/224.
* *NEW*: Rudimentary support for CSS 3 colors.
* *CHANGED*: Attribute selectors are highlighted as @:attribute_name@ instead of @:string@.
* *CHANGED*: Comments are scanned as one token instead of three.
h3. @Scanners::Debug@
* *NEW*: Support for line tokens (@begin_line@ and @end_line@ represented by @[@ and @]@.)
* *FIXED*: Don't send @:error@ and @nil@ tokens for buggy input any more.
* *FIXED*: Closes unclosed tokens at the end of @scan_tokens@.
* *IMPROVED*: Highlight unknown tokens as @:error@.
* *CHANGED*: Raises an error when trying to end an invalid token group.
h3. @Scanners::Delphi@
* *FIXED*: Closes open string groups.
h3. @Scanners::Diff@
* *NEW*: Highlighting of code based on file names.
See ticket "#52":http://redmine.rubychan.de/issues/52.
Use the @:highlight_code@ option to turn this feature off. It's enabled
by default.
This is a very original feature. It enables multi-language highlighting for
diff files, which is especially helpful for CodeRay development itself. The
updated version of the scanner test suite generated .debug.diff.html files
using this.
Note: This is still experimental. Tokens spanning more than one line
may get highlighted incorrectly. CodeRay tries to keep scanner states
between the lines and changes, but the quality of the results depend on
the scanner.
* *NEW*: Inline change highlighting, as suggested by Eric Thomas.
See ticket "#227":http://redmine.rubychan.de/issues/227 for details.
Use the @:inline_diff@ option to turn this feature off. It's enabled by
default.
For single-line changes (that is, a single deleted line followed by a single
inserted line), this feature surrounds the changed parts with an
@:eyecatcher@ group which appears in a more saturated background color.
The implementation is quite complex, and highly experimental. The problem
with multi-layer tokenizing is that the tokens have to be split into parts.
If the inline change starts, say, in the middle of a string, then additional
@:end_group@ and @:begin_group@ tokens must be inserted to keep the group
nesting intact. The extended @Scanner#tokenize@ method and the new
@Tokens#split_into_parts@ method take care of this.
* *NEW*: Highlight the file name in the change headers as @:filename@.
* *CHANGED*: Highlight unknown lines as @:comment@ instead of @:head@.
* *IMPROVED*: Background colors for Diff output have been optimized.
h3. *RENAMED*: @Scanners::ERB@ (was @Scanners::RHTML@)
h3. *NEW*: @Scanners::HAML@
It uses the new :state options of the HTML and Ruby scanners.
Some rare cases are not considered (like @#{...}@ snippets inside of :javascript blocks),
but it highlights pretty well.
h3. @Scanners::HTML@
* *FIXED*: Closes open string groups.
h3. @Scanners::JavaScript@
* *IMPROVED*: Added @NaN@ and @Infinity@ to list of predefined constants.
* *IMPROVED* recognition of RegExp literals with leading spaces.
h3. @Scanners::Java@
* *NEW*: Package names are highlighted as @:namespace@.
See "#210":http://redmine.rubychan.de/issues/210.
h3. *REMOVED*: @Scanners::NitroXHTML@
Nitro is "dead":http://www.nitrohq.com/.
h3. *RENAMED*: @Scanners::Text@ (was @Scanners::Plaintext@)
* *IMPROVED*: Just returns the string without scanning (faster).
This is much faster than scanning until @/\z/@ in Ruby 1.8.
h3. @Scanners::Python@
* *CHANGED*: Docstrings are highlighted as @:comment@.
See "#190":http://redmine.rubychan.de/issues/190.
h3. *NEW*: @Scanners::Raydebug@
Copied from @Scanners::Debug@, highlights the token dump instead of importing it. It also reacts to the @.raydebug@ file
name suffix now.
h3. @Scanners::Ruby@
* *ADDED* more predefined keywords (see http://murfy.de/ruby-constants).
* *IMPROVED* support for singleton method definitions.
See "#147":http://redmine.rubychan.de/issues/147.
* *FIXED*: Don't highlight methods with a capital letter as constants
(eg. @GL.PushMatrix@). GL.PushMatrix).
* *NEW*: Highlight buggy floats (like .5) as @:error@.
* *CLEANUP* of documentation, names of constants and variables, state handling.
Moved @StringState@ class from @patterns.rb@ into a separate file.
* *NEW*: Complicated rule for recognition of @foo=@ style method names.
* *NEW*: Handles @:keep_state@ option (a bit; experimental).
Actually, Ruby checks if there is @[~>=]@, but not @=>@ following the name.
* *REMOVED* @EncodingError@
h3. *REMOVED* @Scanners::Scheme@
* It is too buggy, and nobody was using it. To be added again when it's fixed.
See "#59":http://redmine.rubychan.de/issues/59. *CHANGED*: Does use @:operator@ instead of @:operator_fat@ now.
h3. @Scanners::SQL@
* *IMPROVED*: Extended list of keywords and functions (thanks to
Joshua Galvez, Etienne Massip, and others).
See "#221":http://redmine.rubychan.de/issues/221.
* *FIXED*: Closes open string groups.
* *FIXED*: Words after @.@ are always recognized as @:ident@.
h3. @Scanners::YAML@
* *FIXED*: Allow spaces before colon in mappings.
See "#231":http://redmine.rubychan.de/issues/231.
h3. *NEW*: @Styles::Alpha@
A style that uses transparent HSLA colors as defined in CSS 3.
See "#199":http://redmine.rubychan.de/issues/199.
It also uses the CSS 3 property @user-select: none@ to keep the user from selecting the line numbers. This is especially
nice for @:inline@ line numbers.
See "#226":http://redmine.rubychan.de/issues/226.
h3. @WordList@
Stripped down to 19 LOC.
* *RENAMED* @CaseIgnoringWordList@ to @WordList::CaseIgnoring@.
* *REMOVED* caching option because it creates memory leaks.
* *REMOVED* block option.
h3. @FileType@
* *REMOVED* @FileType#shebang@ is a protected method now.
* *NEW*: Recognizes @.gemspec@, @.rjs@, @.rpdf@ extensions, @Gemfile@, and @Capfile@ as Ruby.
Thanks to the authors of the TextMate Ruby bundle!
* *REMOVED* @FileType#shebang@ is a protected method now.
h3. @Plugin@
* *IMPROVED*: @register_for@ sets the @plugin_id@; it can now be a @Symbol@.
* *ADDED* @PluginHost#const_missing@ method: Plugins are loaded automatically.
Using @Scanners::JavaScript@ in your code loads @scanners/java_script.rb@.
* *ADDED* @#all_plugins@ method to simplify getting
information about all available plugins (suggested by bnhymn).
* *CHANGED* the default plugin key from @nil@ to @:default@.
h3. @GZip@
* *MOVED* into @CodeRay@ namespace.
* *MOVED* file from @gzip_simple.rb@ to @gzip.rb@.
* *REMOVED* @String@ extensions.
h3. More Internal API changes
* *FIXED* @Encoders::HTML#token@'s second parameter is no longer optional.
* *CHANGED* @Encoders::HTML::Output@'s API.
* *REMOVED* lots of unused methods.
h2. Changes in 0.9.8 "banister" [2011-05-01]
Fixes for JRuby's 1.9 mode and minor issues.
h3. Rake tasks
* *REMOVED* obsolete @has_rdoc@ gem specification, fixing a warning.
h3. @Scanners::Scanner@
* *NEW* method @#scan_rest@ replaces @scan_until(/\z/)@, which is broken in JRuby 1.6 --1.9 mode.
See "#297":http://redmine.rubychan.de/issues/297.
h3. @Scanners::CSS@
* *FIXED* LOC counting (should be 0).
See "#296":http://redmine.rubychan.de/issues/296.
h3. @Scanners::Ruby@
* *FIXED* the @IDENT@ pattern not to use character properties, which are broken in JRuby 1.6 --1.9 mode.
See "#297":http://redmine.rubychan.de/issues/297, thanks to banister for reporting!
h3. @Scanners::SQL@
* *ADDED* more keywords: @between@, @databases@, @distinct@, @fields@, @full@, @having@, @is@, @prompt@, @tables@.
See "#221":http://redmine.rubychan.de/issues/221, thanks to Etienne Massip again.
h3. @FileType@
* *NEW* regonizes ColdFusion file type extensions @.cfm@ and @.cfc@ as XML.
See "#298":http://redmine.rubychan.de/issues/298, thanks to Emidio Stani.
h2. Changes in 0.9.7 "Etienne" [2011-01-14]
Fixes a dangerous JavaScript scanner bug, and a testing problem with Ruby 1.9.1.
h3. Tests
* *FIXED* The helper classes were cleaned up; see above functional tests now load the lib directory (instead of the gem) in Ruby 1.9.1.
h3. @Scanners::JavaScript@
* *FIXED* @KEY_CHECK_PATTERN@ regexp
See "#264":http://redmine.rubychan.de/issues/264, thanks to Etienne Massip!
h2. Changes in 0.9.6 "WoNáDo" [2010-11-25]
Minor improvements to the Ruby scanner and a fix for details. Ruby 1.9.
h3. @Scanners::Ruby@
* *IMPROVED* handling of new hash syntax (keys are marked as @:key@ now,
colon is a separate @:operator@ token, all idents can be used as keys)
See "#257":http://code.licenser.net/issues/257, thanks to WoNáDo!
* *ADDED* @__ENCODING__@ magic constant (Ruby 1.9)
* *FIXED*: Scanner no longer tries to modify the input string on Ruby 1.9.
See "#260":http://code.licenser.net/issues/260, thanks to Jan Lelis!
h2. Changes in 0.9.5 "Germany.rb" [2010-09-28]
Support for Rubinius ("#251":http://redmine.rubychan.de/issues/251), improved mutlibyte handling, Ruby 1.9 syntax, and valid HTML.
h3. @Encoders::HTML@
* *FIXED*: Line tokens use @span@ with @display: block@ instead of @div@, which was invalid HTML ("#255":http://redmine.rubychan.de/issues/255).
h3. @Scanner::Scanner@
* *IMPROVED* handling of encodings in Ruby 1.9: UTF-8 and Windows-1252 are checked.
* *NEW*: Invalid chars will be converted to @?@ in Ruby 1.9.
* *FIXED* @string=@ method for Rubinius. See "issue 481":http://github.com/evanphx/rubinius/issues/481 on their site.
h3. @Scanners::CSS@
* *FIXED*: Don't use non-ASCII regexps.
h3. @Scanners::Diff@
* *FIXED*: Highlight unexpected lines as @:comment@.
h3. @Scanners::PHP@
* *FIXED*: Use @ASCII-8BIT@ encoding for now.
h3. @Scanners::Ruby@
* *ADDED* support for some Ruby 1.9 syntax ("#254":http://redmine.rubychan.de/issues/254):
** the @->@ lambda shortcut
** new Hash syntax using colons (@{ a: b }@)
* *FIXED*: Use @UTF-8@ encoding.
* *IMPROVED* unicode support on Ruby 1.8 ("#253":http://redmine.rubychan.de/issues/253).
* *FIXED* recognition of non-ASCII identifiers in Ruby 1.9, JRuby, and Rubinius ("#253":http://redmine.rubychan.de/issues/253).
* *CHANGED* @Plugin@ API heredoc recognition to ignore delimiters starting with a digit. This is incorrect, but causes less false positives.
h3. @Scanners::SQL@
* *FIXED* scanning of comments; nice catch, Rubinius!
("#252":http://redmine.rubychan.de/issues/252)
h2. Changes in 0.9.4 "Ramadan" [2010-08-31]
Updated command line interface and minor scanner fixes for the Diff, HTML, and RHTML scanners.
h3. @coderay@ executable
* *FIXED*: Partly rewritten, simplified, fixed.
("#244":http://redmine.rubychan.de/issues/244)
h3. @Scanners::Diff@
* *FIXED* handling of change headers with code on the same line as the @@ marker.
("#247":http://redmine.rubychan.de/issues/242)
h3. @Scanners::HTML@
* *FIXED* a missing regexp modifier that slowed down the scanning.
("#245":http://redmine.rubychan.de/issues/245)
h3. @Scanners::RHTML@
* *FIXED* highlighting of ERB comment blocks.
("#246":http://redmine.rubychan.de/issues/246)
h2. Changes in 0.9.3 "Eyjafjallajökull" [2010-04-18]
* *FIXED*: Documentation of Tokens.
("#218":http://redmine.rubychan.de/issues/218)
h3. @coderay@ executable
* *NEW*: automatic TTY detection (uses @Term@ encoder)
* *NEW*: optional 3rd parameter for the filename
* *FIXED*: Converted to UNIX format.
* *FIXED*: Warn about generated files.
* *FIXED*: Ensure line break after the output (especially for LoC counter).
h3. @Scanners::JavaScript@
* *FIXED*: Don't keep state of XML scanner between calls for E4X literals.
h3. @Scanners::Java@, @Scanners::JSON@
* *FIXED*: Close unfinished strings with the correct token kind.
h2. Changes in 0.9.2 "Flameeyes" [2010-03-14]
* *NEW* Basic tests and a _Rakefile_ are now included in the Gem. [Flameeyes]
A @doc@ task is also included.
* *FIXED* Use @$CODERAY_DEBUG@ for debugging instead of @$DEBUG@. [Trans]
("#192":http://redmine.rubychan.de/issues/192)
* *REMOVED* @Term::Ansicolor@ was simplified bundled under _lib/_, but not used. [Flameeyes]
("#205":http://redmine.rubychan.de/issues/205)
* *WORKAROUND* for Ruby bug
"#2745":http://redmine.ruby-lang.org/issues/show/2745
h3. @Encoders::Term@
* *FIXED* strings are closed correctly
("#138":http://redmine.rubychan.de/issues/138)
* *FIXED* several token kinds had no associated color
("#139":http://redmine.rubychan.de/issues/139)
* *NEW* alias @terminal@
*NOTE:* This encoder will be renamed to @Encoders::Terminal@ in the next release.
h3. @Scanners::Debug@
* *FIXED* Don't close tokens that are not open. Send @:error@ token instead.
h3. @Scanners::Groovy@
* *FIXED* token kind of closing brackets is @:operator@ instead of @nil@
("#148":http://redmine.rubychan.de/issues/148)
h3. @Scanners::PHP@
* *FIXED* allow @\@ operator (namespace separator)
("#209":http://redmine.rubychan.de/issues/209)
h3. @Scanners::YAML@
* *FIXED* doesn't send debug tokens when @$DEBUG@ is true [Trans]
("#149":http://redmine.rubychan.de/issues/149)
h2. Changes in 0.9.1 [2009-12-31]
h3. Token classes
* *NEW* token classes @:complex@, @:decorator@, @:imaginary@
(all for Python)
* *REMOVED* token class @:procedure@
– use @:function@ or @:method@ instead.
h3. @Tokens@
* *NEW* method @#scanner@
Stores the scanner.
* *REMOVED* methods @.write_token@, @.read_token@, @.escape@, @.unescape@
They were only used by the @Tokens@ encoder, which was removed also.
h3. @Encoders::Encoder@
* *REMOVED* Don't require the _stringio_ library.
* *NEW* public methods @#open_token@, @#close_token@, @#begin_line@, @#end_line@
These methods are called automatically, like @#text_token@.
* *NEW* proteced method @#append_encoded_token_to_output@
h3. @Encoders::Tokens@
* *REMOVED* – use @Tokens#dump@ and stripped @Tokens.load@.
h3. @Encoders::Filter@
* *NEW*
A @Filter@ encoder has another @Tokens@ instance as output.
h3. @Encoders::TokenClassFilter@
* *NEW*
It takes 2 options, @:exclude@ and @:include@, that specify which token classes
to include or exclude for the output. They can be a single token class,
an @Array@ of classes, or the value @:all@.
h3. @Encoders::CommentFilter@
* *NEW*
Removes tokens of the @:comment@ class.
h3. @Encoders::LinesOfCode@
* *NEW*
Counts the lines of code according to the @KINDS_NOT_LOC@ token class list
defined by the scanner. It uses the new @TokenClassFilter@.
Alias: @:loc@, as in @tokens.loc@.
h3. @Encoders::JSON@
* *NEW*
Outputs tokens in a simple JSON format.
h3. @Encoders::Term@
* *NEW* (beta, by Rob Aldred)
Outputs code highlighted for a color terminal.
h3. @Encoders::HTML@
* *NEW* option @:title@ (default value is _CodeRay output_)
Setting this changes the title of the HTML page.
* *NEW* option @:highlight_lines@ (default: @nil@)
Highlights the given set of line numbers.
- *REMOVED* option @:level@
It didn't do anything. CodeRay always outputs XHTML.
h3. @Encoders::Text@
* Uses @Encoder@ interface with @super@ and @#text_token@.
h3. @Encoders::XML@
* @FIXED@ ("#94":http://redmine.rubychan.de/issues/94)
It didn't work at all.
h3. Scanners
* *NEW* Mapped @:h@ to @:c@, @:cplusplus@ and @:'c++'@ to @:cpp@,
@:ecma@, @:ecmascript@, @:ecma_script@ to @:java_script@,
@:pascal@ to @:delphi@, and @:plain@ to @:plaintext@.
h3. @Scanners::Scanner@
* *NEW* constant @KINDS_NOT_LOC@
A list of all unnecessary features. token classes not considered in LOC count.
Added appropriate values for scanners.
* *CHANGED* Moved @GZip@ *NEW* method @#lang@ returns the scanner's lang, which is its @plugin_id@.
* *FIXED* automatic, safe UTF-8 detection _[Ruby 1.9]_
* *FIXED* column takes care of multibyte encodings _[Ruby 1.9]_
* *FIXED* is dumpable (@Tokens@ store their scanner in an @@scanner@ variable)
h3. @Scanners::Cpp@
* *NEW* (C++)
h3. @Scanners::Groovy@
* *NEW* (beta)
h3. @Scanners::Python@
* *NEW*
h3. @Scanners::PHP@
* *NEW* (based on Stefan Walk's work)
h3. @Scanners::SQL@
* *NEW* (based on code by Josh Goebel)
h3. @Scanners::C@
* *IMPROVED* added a list of @:directive@ tokens that were @:reserved@ before
* *IMPROVED* detection of labels
* *IMPROVED* allow @1L@ and @1LL@ style literals
h3. @Scanners::CSS@
* *IMPROVED* element selectors are highlighted as @:type@ instead of @:keyword@
h3. @Scanners::Delphi@
* *IMPROVED* Don't cache tokens in CaseIgnoringWordList.
h3. @Scanners::Java@
* *IMPROVED* @assert@ is highlighted as a @:keyword@ now
* *IMPROVED* @const@ and @goto@ are highlighted as @:reserved@
* *IMPROVED* @false@, @true@, and @null@ are highlighted as @:pre_constant@
* *IMPROVED* @threadsafe@ is no longer a @:directive@
* *IMPROVED* @String@ is highlighted as a @:pre_type@
* *IMPROVED* built-in classes ending with _Error_ or _Exception_ are
highlighted as a @:exception@ instead of @:pre_type@
h3. @Scanners::JavaScript@
* *NEW* a list of @PREDEFINED_CONSTANTS@ to be highlighted as @:pre_constant@
* *NEW* XML literals are recognized and highlighted
* *NEW* function name highlighting
* *IMPROVED* @.1@ is highlighted a number
* *FIXED* strings close with the correct kind when terminated unexpectedly
h3. @Scanners::JSON@
* *IMPROVED* constants (@true@, @false@, @nil@) are highlighted as @:value@
h3. @Scanners::Ruby@
* *IMPROVED* @Patterns::KEYWORDS_EXPECTING_VALUE@ for more accurate
@value_expected@ detection
* *IMPROVED* handling of @\@ as a string delimiter
* *IMPROVED* handling of unicode strings; automatic switching to unicode
* *IMPROVED* highlighting of @self.method@ definitions
* *REMOVED* @Patterns::FANCY_START_SAVE@ (obsolete)
* *FIXED* encoding issues _[Ruby 1.9]_
* *FIXED* a problem in early Ruby 1.8.6 patch versions with @Regexp.escape@
h3. @Scanners::YAML@
* *IMPROVED* indentation detection
h3. @Styles::Cycnus@
* changed a few colors (exceptions, inline strings, predefined types)
h3. @Plugin@
* *NEW* method @#title@
Set and get the plugin's title. Titles can be arbitrary strings.
* *NEW* method @#helper@ loads helpers from different plugins
Use this syntax: @helper 'other_plugin/helper_name'@
h3. @FileType@ libraries into @CodeRay@; cleaned them up.
* *NEW* @FileType[]@ takes @Pathname@ instances
* *NEW* regonizes @.cc@, @.cpp@, @.cp@, @.cxx@, @.c++@, @.C@, @.hh@, @.hpp@, @.h++@, @.cu@ extensions (C++)
Thanks to Sander Cox and the TextMate C bundle.
* *NEW* regonizes @.pas@, @.dpr@ extensions (Delphi)
* *NEW* regonizes @.gvy@, @.groovy@ extensions (Groovy)
* *NEW* regonizes @.php@, @.php3@, @.php4@, @.php5@ extensions (PHP)
* *NEW* regonizes @.py@, @.py3@, @.pyw@ extensions (Python)
* *NEW* regonizes @.rxml@ extension (Ruby)
* *NEW* regonizes @.sql@ extension (SQL)
* File types list was sorted alphabetically.
h3. @CaseIgnoringWordList@
* *FIXED* ("#97":http://redmine.rubychan.de/issues/97)
The default value is no longer ignored.
h3. @ForRedCloth@
* *FIXED* for RedCloth versions 4.2.0+ ("#119":http://redmine.rubychan.de/issues/119)
h3. Cleanups
* warnings about character classes _[Ruby 1.9]_
* encoding issues _[Ruby 1.9]_
* documentation, code