Import syntax highlighting docs
This commit is contained in:
Garen Torikian
parent
2ad5b6e97b
commit
31ff93650c
2 changed files with 408 additions and 61 deletions
|
|
@ -387,8 +387,8 @@ UL.menu-footer LI A:hover {
|
|||
}
|
||||
|
||||
.nav-tabs > li > a, .nav-pills > li > a {
|
||||
padding-right: 30px;
|
||||
padding-left: 30px;
|
||||
padding-right: 17px;
|
||||
padding-left: 17px;
|
||||
border-right: 1px solid #bbb;
|
||||
border-radius: 0;
|
||||
margin: 0;
|
||||
|
|
@ -403,8 +403,8 @@ UL.menu-footer LI A:hover {
|
|||
}
|
||||
|
||||
.nav.nav-pills li:first-child > a {
|
||||
padding-left: 29px;
|
||||
padding-right: 29px;
|
||||
padding-left: 28px;
|
||||
padding-right: 28px;
|
||||
}
|
||||
|
||||
.nav.nav-pills li:first-child > a > img {
|
||||
|
|
|
|||
461
index.html
461
index.html
|
|
@ -47,6 +47,9 @@
|
|||
<li>
|
||||
<a href="#howto" data-toggle="tab">How-To Guide</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="#higlighter" data-toggle="tab">Syntax Highlighters</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="#api">API Reference</a>
|
||||
</li>
|
||||
|
|
@ -342,64 +345,408 @@ editor.replace('bar');</code></pre>
|
|||
//...
|
||||
}
|
||||
});</code></pre>
|
||||
<h3 id ="importing">Importing Themes and Languages</h3>
|
||||
<p>ACE supports the importing of <em>.tmtheme</em> and <em>.tmlanguage</em> files for use
|
||||
in the editor. The task is accomplished by two simple node scripts.</p>
|
||||
|
||||
<h4>Importing Textmate/Sublime Themes</h4>
|
||||
<p>To import a <em>.tmtheme</em> file into ACE:</p>
|
||||
<ol>
|
||||
<li> Go to the <em>tool</em> folder, and run <code>npm install</code> to install required
|
||||
dependencies.</li>
|
||||
<li> Drop your <em>.tmtheme</em> file into the <em>tmthemes folder</em>.</li>
|
||||
<li> Update the <a href="https://github.com/ajaxorg/ace/blob/master/tool/tmtheme.js#L180">tmtheme.js</a> file to include your new theme.</li>
|
||||
<li> Run <code>node tmtheme.js</code>
|
||||
</li>
|
||||
</ol><p>Your <em>.tmtheme</em> will be converted and placed into <em>lib/ace/theme</em>
|
||||
alongside other themes. Note that there’s one more class we’ve added
|
||||
that isn’t available in regular Textmate themes, and that’s for
|
||||
<code>.ace_indent-guide</code>. This class adds indentation guides for your theme,
|
||||
using a base64-encoded png. In general, the dark themes and light themes
|
||||
each have their own strings, so you can just copy the class from an
|
||||
equivalent theme.</p>
|
||||
|
||||
<h4>Importing Textmate/Sublime Languages</h4>
|
||||
|
||||
<p>If you’re interested in porting over an existing <em>.tmlanguage</em> file into
|
||||
Ace’s mode syntax highlighting, there’s a tool to accomplish that, too.</p>
|
||||
|
||||
<ol>
|
||||
<li> Go to the <em>tool</em> folder. Run <code>npm install</code> to install required
|
||||
dependencies.</li>
|
||||
<li> Drop your <em>.tmlanguage</em> file into the tools folder.</li>
|
||||
<li> Run <code>node tmlanguage.js <path_to_tmlanguage_file></code>, such as
|
||||
<code>node tmlanguage.js MyGreatLanguage.tmlanguage</code>.</li>
|
||||
</ol><p>Your <em>.tmlanguage</em> file will be converted to the best of the converter’s
|
||||
ability. It is an understatement to say that the tool is imperfect.
|
||||
Probably, this will never be able to be fully autogenerated. In
|
||||
<em>.tmlanguage</em> files, for instance, one sees the use of
|
||||
lookbehinds/negative lookbehinds, which JavaScript regexps simply do not
|
||||
have. There’s a list of other non-determinable items, too:</p>
|
||||
|
||||
<ul>
|
||||
<li> Deciding which state to transition to<br>
|
||||
While the tool does create new states correctly, it labels them
|
||||
with generic terms like <code>state_2</code>, <code>state_10</code> <em>e.t.c.</em>
|
||||
</li>
|
||||
<li> Extending modes<br>
|
||||
Many modes say something like <code>include source.c</code>, to mean, “add all
|
||||
the rules in C highlighting.” That syntax does not make sense to Ace
|
||||
or this tool</li>
|
||||
<li> Rule preference order</li>
|
||||
<li> Gathering keywords<br>
|
||||
Most likely, you’ll need to take keywords from your language file
|
||||
and run them through <code>createKeywordMapper()</code>
|
||||
</li>
|
||||
</ul><p>Two files are created and placed in <em>lib/ace/mode</em>: one for the language
|
||||
mode, and one for the set of highlight rules. You will still need to add
|
||||
the code into <em>kitchen_sink.html</em> and <em>demo.js</em>, as well as write any
|
||||
tests for the highlighting.</p>
|
||||
</div>
|
||||
|
||||
<div class="tab-pane fade" id="higlighter">
|
||||
<h1>Creating a Syntax Highlighter for Ace</h1><p>Creating a new syntax highlighter for Ace is extremly simple. You'll need to define two pieces of code: a new mode, and a new set of highlighting rules.</p>
|
||||
<h2 id='where-to-start'><a class='heading_anchor' href='#where-to-start'><i class='headerLinkIcon'></i></a>Where to Start</h2>
|
||||
<p>We recommend using the the <a href="http://ace.ajax.org/tool/mode_creator.html">Ace Mode Creator</a> when defining your highlighter. This allows you to inspect your code's tokens, as well as providing a live preview of the syntax highlighter in action.</p>
|
||||
<h2 id='defining-a-mode'><a class='heading_anchor' href='#defining-a-mode'><i class='headerLinkIcon'></i></a>Defining a Mode</h2>
|
||||
<p>Every language needs a mode. A mode contains the paths to a language's syntax highlighting rules, indentation rules, and code folding rules. Without defining a mode, Ace won't know anything about the finer aspects of your language.</p>
|
||||
<p>Here is the starter template we'll use to create a new mode:</p>
|
||||
<pre><code class="language-javascript">define(<span class="keyword">function</span>(require, exports, module) {
|
||||
<span class="string">"use strict"</span>;
|
||||
|
||||
<span class="keyword">var</span> oop = require(<span class="string">"../lib/oop"</span>);
|
||||
<span class="comment">// defines the parent mode</span>
|
||||
<span class="keyword">var</span> TextMode = require(<span class="string">"./text"</span>).Mode;
|
||||
<span class="keyword">var</span> Tokenizer = require(<span class="string">"../tokenizer"</span>).Tokenizer;
|
||||
<span class="keyword">var</span> MatchingBraceOutdent = require(<span class="string">"./matching_brace_outdent"</span>).MatchingBraceOutdent;
|
||||
|
||||
<span class="comment">// defines the language specific highlighters and folding rules</span>
|
||||
<span class="keyword">var</span> MyNewHighlightRules = require(<span class="string">"./mynew_highlight_rules"</span>).MyNewHighlightRules;
|
||||
<span class="keyword">var</span> MyNewFoldMode = require(<span class="string">"./folding/mynew"</span>).MyNewFoldMode;
|
||||
|
||||
<span class="keyword">var</span> Mode = <span class="keyword">function</span>() {
|
||||
<span class="comment">// set everything up</span>
|
||||
<span class="keyword">var</span> highlighter = <span class="keyword">new</span> MyNewHighlightRules();
|
||||
<span class="keyword">this</span>.$outdent = <span class="keyword">new</span> MatchingBraceOutdent();
|
||||
<span class="keyword">this</span>.foldingRules = <span class="keyword">new</span> MyNewFoldMode();
|
||||
|
||||
<span class="keyword">this</span>.$tokenizer = <span class="keyword">new</span> Tokenizer(highlighter.getRules());
|
||||
};
|
||||
oop.inherits(Mode, TextMode);
|
||||
|
||||
(<span class="keyword">function</span>() {
|
||||
<span class="comment">// Extra logic goes here--we won't be covering all of this</span>
|
||||
|
||||
<span class="comment">/* These are all optional pieces of code!
|
||||
this.getNextLineIndent = function(state, line, tab) {
|
||||
var indent = this.$getIndent(line);
|
||||
return indent;
|
||||
};
|
||||
|
||||
this.checkOutdent = function(state, line, input) {
|
||||
return this.$outdent.checkOutdent(line, input);
|
||||
};
|
||||
|
||||
this.autoOutdent = function(state, doc, row) {
|
||||
this.$outdent.autoOutdent(doc, row);
|
||||
};
|
||||
|
||||
this.createWorker = function(session) {
|
||||
var worker = new WorkerClient(["ace"], "ace/mode/mynew_worker", "NewWorker");
|
||||
worker.attachToDocument(session.getDocument());
|
||||
|
||||
return worker;
|
||||
};
|
||||
*/</span>
|
||||
}).call(Mode.prototype);
|
||||
|
||||
exports.Mode = Mode;
|
||||
});</code></pre>
|
||||
<p>What's going on here? First, you're defining the path to <code>TextMode</code> (more on this later). Then you're pointing the mode to your definitions for the highlighting rules, as well as your rules for code folding. Finally, you're setting everything up to find those rules, and exporting the Mode so that it can be consumed. That's it!</p>
|
||||
<p>Regarding <code>TextMode</code>, you'll notice that it's only being used once: <code>oop.inherits(Mode, TextMode);</code>. If your new language depends on the rules of another language, you can choose to inherit the same rules, while expanding on it with your language's own requirements. For example, PHP inherits from HTML, since it can be embedded directly inside <em>.html</em> pages. You can either inherit from <code>TextMode</code>, or any other existing mode, if it already relates to your language.</p>
|
||||
<p>All Ace modes can be found in the <em>lib/ace/mode</em> folder.</p>
|
||||
<h2 id='defining-syntax-highlighting-rules'><a class='heading_anchor' href='#defining-syntax-highlighting-rules'><i class='headerLinkIcon'></i></a>Defining Syntax Highlighting Rules</h2>
|
||||
<p>The Ace highlighter can be considered to be a state machine. Regular expressions define the tokens for the current state, as well as the transitions into another state. Let's define <em>mynew_highlight_rules.js</em>, which our mode above uses.</p>
|
||||
<p>All syntax highlighters start off looking something like this:</p>
|
||||
<pre><code class="language-javascript">define(<span class="keyword">function</span>(require, exports, module) {
|
||||
<span class="string">"use strict"</span>;
|
||||
|
||||
<span class="keyword">var</span> oop = require(<span class="string">"../lib/oop"</span>);
|
||||
<span class="keyword">var</span> TextHighlightRules = require(<span class="string">"./text_highlight_rules"</span>).TextHighlightRules;
|
||||
|
||||
<span class="keyword">var</span> MyNewHighlightRules = <span class="keyword">function</span>() {
|
||||
|
||||
<span class="comment">// regexp must not have capturing parentheses. Use (?:) instead.</span>
|
||||
<span class="comment">// regexps are ordered -> the first match is used</span>
|
||||
<span class="keyword">this</span>.$rules = {
|
||||
<span class="string">"start"</span> : [
|
||||
{
|
||||
token: <span class="xml"><span class="tag"><<span class="title">token</span>></span>, // String, Array, or Function: the CSS token to apply
|
||||
regex: <span class="tag"><<span class="title">regex</span>></span>, // String or RegExp: the regexp to match
|
||||
next: <span class="tag"><<span class="title">next</span>></span> // [Optional] String: next state to enter
|
||||
}
|
||||
]
|
||||
};
|
||||
};
|
||||
|
||||
oop.inherits(MyNewHighlightRules, TextHighlightRules);
|
||||
|
||||
exports.MyNewHighlightRules = MyNewHighlightRules;
|
||||
|
||||
});</span></code></pre>
|
||||
<p>The token state machine operates on whatever is defined in <code>this.$rules</code>. The highlighter always begins at the <code>start</code> state, and progresses down the list, looking for a matching <code>regex</code>. When one is found, the resulting text is wrapped within a <code><span class="ace_<token>"></code> tag, where <code><token></code> is defined as the <code>token</code> property. Note that all tokens are preceded by the <code>ace_</code> prefix when they're rendered on the page.</p>
|
||||
<p>Once again, we're inheriting from <code>TextHighlightRules</code> here. We could choose to make this any other language set we want, if our new language requires previously defined syntaxes. For more information on extending languages, see "<a href="#extending-highlighters">extending Highlighters</a>" below.</p>
|
||||
<h3 id='defining-tokens'><a class='heading_anchor' href='#defining-tokens'><i class='headerLinkIcon'></i></a>Defining Tokens</h3>
|
||||
<p>The Ace highlighting system is heavily inspired by the <a href="http://manual.macromates.com/en/language_grammars">TextMate language grammar</a>. Most tokens will follow the conventions of TextMate when naming grammars. A thorough (albeit incomplete) list of tokens can be found <a href="https://github.com/ajaxorg/ace/wiki/Creating-or-Extending-an-Edit-Mode#wiki-commonTokens">on the Ace Wiki</a>.</p>
|
||||
<p>For the complete list of tokens, see <em><a href="https://github.com/ajaxorg/ace/blob/master/tool/tmtheme.js">tool/tmtheme.js</a></em>. It is possible to add new token names, but the scope of that knowledge is outside of this document.</p>
|
||||
<p>Multiple tokens can be applied to the same text by adding dots in the token, <em>e.g.</em> <code>token: support.function</code> wraps the text in a <code><span class="ace_support ace_function"></code> tag.</p>
|
||||
<h3 id='defining-regular-expressions'><a class='heading_anchor' href='#defining-regular-expressions'><i class='headerLinkIcon'></i></a>Defining Regular Expressions</h3>
|
||||
<p>Regular expressions can either be a RegExp or String definition</p>
|
||||
<p>If you're using a regular expression, remember to start and end the line with the <code>/</code> character, like this:</p>
|
||||
<pre><code class="language-javascript">{
|
||||
token : <span class="string">"constant.language.escape"</span>,
|
||||
regex : <span class="regexp">/\$[\w\d]+/</span>
|
||||
}</code></pre>
|
||||
<p>A caveat of using stringed regular expressions is that any <code>\</code> character must be escaped. That means that even an innocuous regular expression like this:</p>
|
||||
<pre><code class="language-javascript">regex: <span class="string">"function\s*\(\w+\)"</span></code></pre>
|
||||
<p>Must actually be written like this:</p>
|
||||
<pre><code class="language-javascript">regex: <span class="string">"function\\s*\(\\w+\)"</span></code></pre>
|
||||
<h4 id='groupings'><a class='heading_anchor' href='#groupings'><i class='headerLinkIcon'></i></a>Groupings</h4>
|
||||
<p>You can also include flat regexps--<code>(var)</code>--or have matching groups--<code>((a+)(b+))</code>. There is a strict requirement whereby matching groups <strong>must</strong> cover the entire matched string; thus, <code>(hel)lo</code> is invalid. If you want to create a non-matching group, simply start the group with the <code>?:</code> predicate; thus, <code>(hel)(?:lo)</code> is okay. You can, of course, create longer non-matching groups. For example:</p>
|
||||
<pre><code class="language-javascript">{
|
||||
token : <span class="string">"constant.language.boolean"</span>,
|
||||
regex : <span class="regexp">/(?:true|false)\b/</span>
|
||||
},</code></pre>
|
||||
<p>For flat regular expression matches, <code>token</code> can be a String, or a Function that takes a single argument (the match) and returns a string token. For example, using a function might look like this:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">var</span> colors = lang.arrayToMap(
|
||||
(<span class="string">"aqua|black|blue|fuchsia|gray|green|lime|maroon|navy|olive|orange|"</span> +
|
||||
<span class="string">"purple|red|silver|teal|white|yellow"</span>).split(<span class="string">"|"</span>)
|
||||
);
|
||||
|
||||
<span class="keyword">var</span> fonts = lang.arrayToMap(
|
||||
(<span class="string">"arial|century|comic|courier|garamond|georgia|helvetica|impact|lucida|"</span> +
|
||||
<span class="string">"symbol|system|tahoma|times|trebuchet|utopia|verdana|webdings|sans-serif|"</span> +
|
||||
<span class="string">"serif|monospace"</span>).split(<span class="string">"|"</span>)
|
||||
);
|
||||
|
||||
...
|
||||
|
||||
{
|
||||
token: <span class="keyword">function</span>(value) {
|
||||
<span class="keyword">if</span> (colors.hasOwnProperty(value.toLowerCase())) {
|
||||
<span class="keyword">return</span> <span class="string">"support.constant.color"</span>;
|
||||
}
|
||||
<span class="keyword">else</span> <span class="keyword">if</span> (fonts.hasOwnProperty(value.toLowerCase())) {
|
||||
<span class="keyword">return</span> <span class="string">"support.constant.fonts"</span>;
|
||||
}
|
||||
<span class="keyword">else</span> {
|
||||
<span class="keyword">return</span> <span class="string">"text"</span>;
|
||||
}
|
||||
},
|
||||
regex: <span class="string">"\\-?[a-zA-Z_][a-zA-Z0-9_\\-]*"</span>
|
||||
}</code></pre>
|
||||
<p>If <code>token</code> is a function,it should take the same number of arguments as there are groups, and return an array of tokens.</p>
|
||||
<p>For grouped regular expressions, <code>token</code> can be a String, in which case all matched groups are given that same token, like this:</p>
|
||||
<pre><code class="language-javascript">{
|
||||
token: <span class="string">"identifier"</span>,
|
||||
regex: <span class="string">"(\\w+\\s*:)(\\w*)"</span>
|
||||
}</code></pre>
|
||||
<p>More commonly, though, <code>token</code> is an Array (of the same length as the number of groups), whereby matches are given the token of the same alignment as in the match. For a complicated regular expression, like defining a function, that might look something like this:</p>
|
||||
<pre><code class="language-javascript">{
|
||||
token : [<span class="string">"storage.type"</span>, <span class="string">"text"</span>, <span class="string">"entity.name.function"</span>],
|
||||
regex : <span class="string">"(function)(\\s+)([a-zA-Z_][a-zA-Z0-9_]*\\b)"</span>
|
||||
}</code></pre>
|
||||
<h2 id='defining-states'><a class='heading_anchor' href='#defining-states'><i class='headerLinkIcon'></i></a>Defining States</h2>
|
||||
<p>The syntax highlighting state machine stays in the <code>start</code> state, until you define a <code>next</code> state for it to advance to. At that point, the tokenizer stays in that new <code>state</code>, until it advances to another state. Afterwards, you should return to the original <code>start</code> state.</p>
|
||||
<p>Here's an example:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">this</span>.$rules = {
|
||||
<span class="string">"start"</span> : [ {
|
||||
token : <span class="string">"text"</span>,
|
||||
merge : <span class="literal">true</span>,
|
||||
regex : <span class="string">"<\\!\\[CDATA\\["</span>,
|
||||
next : <span class="string">"cdata"</span>
|
||||
},
|
||||
|
||||
<span class="string">"cdata"</span> : [ {
|
||||
token : <span class="string">"text"</span>,
|
||||
regex : <span class="string">"\\]\\]>"</span>,
|
||||
next : <span class="string">"start"</span>
|
||||
}, {
|
||||
token : <span class="string">"text"</span>,
|
||||
merge : <span class="literal">true</span>,
|
||||
regex : <span class="string">"\\s+"</span>
|
||||
}, {
|
||||
token : <span class="string">"text"</span>,
|
||||
merge : <span class="literal">true</span>,
|
||||
regex : <span class="string">".+"</span>
|
||||
} ]
|
||||
};</code></pre>
|
||||
<p>In this extremly short sample, we're defining some highlighting rules for when Ace detectes a <code><![CDATA</code> tag. When one is encountered, the tokenizer moves from <code>start</code> into the <code>cdata</code> state. It remains there, applying the <code>text</code> token to any string it encounters. Finally, when it hits a closing <code>]></code> symbol, it returns to the <code>start</code> state and continues to tokenize anything else.</p>
|
||||
|
||||
<h2>Using the TMLanguage Tool</h2>
|
||||
<p>There is a tool that
|
||||
will take an existing <em>tmlanguage</em> file and do its best to convert it into Javascript for Ace to consume. Here's what you need to get started:</p>
|
||||
<ol>
|
||||
<li>In the Ace repository, navigate to the <em><a href="https://github.com/ajaxorg/ace/tree/master/tool">tools</a></em> folder.</li>
|
||||
<li>Run <code>npm install</code> to install required dependencies.</li>
|
||||
<li>Run <code>node tmlanguage.js <path_to_tmlanguage_file></code>; for example, <code>node <path_to_tmlanguage_file> /Users/Elrond/elven.tmLanguage</code></li>
|
||||
</ol>
|
||||
<p>Two files are created and placed in <em>lib/ace/mode</em>: one for the language mode, and one for the set of highlight rules. You will still need to add the code into <em>kitchen_sink.html</em> and <em>demo.js</em>, as well as write any tests for the highlighting.</p>
|
||||
<h3 id='a-note-on-accuracy'><a class='heading_anchor' href='#a-note-on-accuracy'><i class='headerLinkIcon'></i></a>A Note on Accuracy</h3>
|
||||
<p>Your <em>.tmlanguage</em> file will then be converted to the best of the converter’s ability. It is an understatement to say that the tool is imperfect. Probably, language mode creation will never be able to be fully autogenerated. There's a list of non-determinable items; for example:</p>
|
||||
<ul>
|
||||
<li>The use of regular expression lookbehinds<br>This is a concept that JavaScript simply does not have and needs to be faked</li>
|
||||
<li>Deciding which state to transition to<br>While the tool does create new states correctly, it labels them with generic terms like <code>state_2</code>, <code>state_10</code>, <em>e.t.c.</em></li>
|
||||
<li>Extending modes<br>Many modes say something like <code>include source.c</code>, to mean, “add all the rules in C highlighting.” That syntax does not make sense to Ace or this tool (though of course you can <a href="/extension-development-resources/guides/extending_highlighters.html">extending existing highlighters</a>).</li>
|
||||
<li>Rule preference order</li>
|
||||
<li>Gathering keywords<br>Most likely, you’ll need to take keywords from your language file and run them through <code>createKeywordMapper()</code></li>
|
||||
</ul>
|
||||
<p>However, the tool is an excellent way to get a quick start, if you already possess a <em>tmlanguage</em> file for you language.</p>
|
||||
|
||||
<h2>Extending Highlighters</h2>
|
||||
|
||||
<p>Suppose you're working on a LuaPage, PHP embedded in HTML, or a Django template. You'll need to create a syntax highlighter that takes all the rules from the original language (Lua, PHP, or Python) and extends it with some additional identifiers (<code><?lua</code>, <code><?php</code>, <code>{%</code>, for example). Ace allows you to easily extend a highlighter using a few helper functions.</p>
|
||||
<h3 id='getting-existing-rules'><a class='heading_anchor' href='#getting-existing-rules'><i class='headerLinkIcon'></i></a>Getting Existing Rules</h3>
|
||||
<p>To get the existing syntax highlighting rules for a particular language, use the <code>getRules()</code> function. For example:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">var</span> HtmlHighlightRules = require(<span class="string">"./html_highlight_rules"</span>).HtmlHighlightRules;
|
||||
|
||||
<span class="keyword">this</span>.$rules = <span class="keyword">new</span> HtmlHighlightRules().getRules();
|
||||
|
||||
<span class="comment">/*
|
||||
this.$rules == Same this.$rules as HTML highlighting
|
||||
*/</span></code></pre>
|
||||
<h3 id='extending-a-highlighter'><a class='heading_anchor' href='#extending-a-highlighter'><i class='headerLinkIcon'></i></a>Extending a Highlighter</h3>
|
||||
<p>The <code>addRules</code> method does one thing, and it does one thing well: it adds new rules to an existing rule set, and prefixes any state with a given tag. For example, let's say you've got two sets of rules, defined like this:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">this</span>.$rules = {
|
||||
<span class="string">"start"</span>: [ <span class="comment">/* ... */</span> ]
|
||||
};
|
||||
|
||||
<span class="keyword">var</span> newRules = {
|
||||
<span class="string">"start"</span>: [ <span class="comment">/* ... */</span> ]
|
||||
}</code></pre>
|
||||
<p>If you want to incorporate <code>newRules</code> into <code>this.$rules</code>, you'd do something like this:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">this</span>.addRules(newRules, <span class="string">"new-"</span>);
|
||||
|
||||
<span class="comment">/*
|
||||
this.$rules = {
|
||||
"start": [ ... ],
|
||||
"new-start": [ ... ]
|
||||
};
|
||||
*/</span></code></pre>
|
||||
<h3 id='extending-two-highlighters'><a class='heading_anchor' href='#extending-two-highlighters'><i class='headerLinkIcon'></i></a>Extending Two Highlighters</h3>
|
||||
<p>The last function available to you combines both of these concepts, and it's called <code>embedRules</code>. It takes three parameters:</p>
|
||||
<ol>
|
||||
<li>An existing rule set to embed with</li>
|
||||
<li>A prefix to apply for each state in the existing rule set</li>
|
||||
<li>A set of new states to add</li>
|
||||
</ol>
|
||||
<p>Like <code>addRules</code>, <code>embedRules</code> adds on to the existing <code>this.$rules</code> object. </p>
|
||||
<p>To explain this visually, let's take a look at the syntax highlighter for Lua pages, which combines all of these concepts:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">var</span> HtmlHighlightRules = require(<span class="string">"./html_highlight_rules"</span>).HtmlHighlightRules;
|
||||
<span class="keyword">var</span> LuaHighlightRules = require(<span class="string">"./lua_highlight_rules"</span>).LuaHighlightRules;
|
||||
|
||||
<span class="keyword">var</span> LuaPageHighlightRules = <span class="keyword">function</span>() {
|
||||
<span class="keyword">this</span>.$rules = <span class="keyword">new</span> HtmlHighlightRules().getRules();
|
||||
|
||||
<span class="keyword">for</span> (<span class="keyword">var</span> i <span class="keyword">in</span> <span class="keyword">this</span>.$rules) {
|
||||
<span class="keyword">this</span>.$rules[i].unshift({
|
||||
token: <span class="string">"keyword"</span>,
|
||||
regex: <span class="string">"<\\%\\=?"</span>,
|
||||
next: <span class="string">"lua-start"</span>
|
||||
}, {
|
||||
token: <span class="string">"keyword"</span>,
|
||||
regex: <span class="string">"<\\?lua\\=?"</span>,
|
||||
next: <span class="string">"lua-start"</span>
|
||||
});
|
||||
}
|
||||
<span class="keyword">this</span>.embedRules(LuaHighlightRules, <span class="string">"lua-"</span>, [
|
||||
{
|
||||
token: <span class="string">"keyword"</span>,
|
||||
regex: <span class="string">"\\%>"</span>,
|
||||
next: <span class="string">"start"</span>
|
||||
},
|
||||
{
|
||||
token: <span class="string">"keyword"</span>,
|
||||
regex: <span class="string">"\\?>"</span>,
|
||||
next: <span class="string">"start"</span>
|
||||
}
|
||||
]);
|
||||
};</code></pre>
|
||||
<p>Here, <code>this.$rules</code> starts off as a set of HTML highlighting rules. To this set, we add two new checks for <code><%=</code> and <code><?lua=</code>. We also delegate that if one of these rules are matched, we should move onto the <code>lua-start</code> state. Next, <code>embedRules</code> takes the already existing set of <code>LuaHighlightRules</code> and applies the <code>lua-</code> prefix to each state there. Finally, it adds two new checks for <code>%></code> and <code>?></code>, allowing the state machine to return to <code>start</code>.</p>
|
||||
|
||||
<h3>Code Folding</h3>
|
||||
|
||||
<p>Adding new folding rules to your mode can be a little tricky. First, insert the following lines of code into your mode definition:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">var</span> MyFoldMode = require(<span class="string">"./folding/newrules"</span>).FoldMode;
|
||||
|
||||
...
|
||||
<span class="keyword">var</span> MyMode = <span class="keyword">function</span>() {
|
||||
|
||||
...
|
||||
|
||||
<span class="keyword">this</span>.foldingRules = <span class="keyword">new</span> MyFoldMode();
|
||||
};</code></pre>
|
||||
<p>You'll be defining your code folding rules into the <em>lib/ace/mode/folding</em> folder. Here's a template that you can use to get started:</p>
|
||||
<pre><code class="language-javascript">define(<span class="keyword">function</span>(require, exports, module) {
|
||||
<span class="string">"use strict"</span>;
|
||||
|
||||
<span class="keyword">var</span> oop = require(<span class="string">"../../lib/oop"</span>);
|
||||
<span class="keyword">var</span> Range = require(<span class="string">"../../range"</span>).Range;
|
||||
<span class="keyword">var</span> BaseFoldMode = require(<span class="string">"./fold_mode"</span>).FoldMode;
|
||||
|
||||
<span class="keyword">var</span> FoldMode = exports.FoldMode = <span class="keyword">function</span>() {};
|
||||
oop.inherits(FoldMode, BaseFoldMode);
|
||||
|
||||
(<span class="keyword">function</span>() {
|
||||
|
||||
<span class="comment">// regular expressions that identify starting and stopping points</span>
|
||||
<span class="keyword">this</span>.foldingStartMarker;
|
||||
<span class="keyword">this</span>.foldingStopMarker;
|
||||
|
||||
<span class="keyword">this</span>.getFoldWidgetRange = <span class="keyword">function</span>(session, foldStyle, row) {
|
||||
<span class="keyword">var</span> line = session.getLine(row);
|
||||
|
||||
<span class="comment">// test each line, and return a range of segments to collapse</span>
|
||||
};
|
||||
|
||||
}).call(FoldMode.prototype);
|
||||
|
||||
});</code></pre>
|
||||
<p>Just like with <code>TextMode</code> for syntax highlighting, <code>BaseFoldMode</code> contains the starting point for code folding logic. <code>foldingStartMarker</code> defines your opening folding point, while <code>foldingStopMarker</code> defines the stopping point. For example, for a C-style folding system, these values might look like this:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">this</span>.foldingStartMarker = <span class="regexp">/(\{|\[)[^\}\]]*$|^\s*(\/\*)/</span>;
|
||||
<span class="keyword">this</span>.foldingStopMarker = <span class="regexp">/^[^\[\{]*(\}|\])|^[\s\*]*(\*\/)/</span>;</code></pre>
|
||||
<p>These regular expressions identify various symbols--<code>{</code>, <code>[</code>, <code>//</code>--to pay attention to. <code>getFoldWidgetRange</code> matches on these regular expressions, and when found, returns the range of relevant folding points. For more information on the <code>Range</code> object, see <a href="http://ace.ajax.org/#nav=api&api=range">the Ace API documentation</a>.</p>
|
||||
<p>Again, for a C-style folding mechanism, a range to return for the starting fold might look like this:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">var</span> line = session.getLine(row);
|
||||
<span class="keyword">var</span> match = line.match(<span class="keyword">this</span>.foldingStartMarker);
|
||||
<span class="keyword">if</span> (match) {
|
||||
<span class="keyword">var</span> i = match.index;
|
||||
|
||||
<span class="keyword">if</span> (match[<span class="number">1</span>])
|
||||
<span class="keyword">return</span> <span class="keyword">this</span>.openingBracketBlock(session, match[<span class="number">1</span>], row, i);
|
||||
|
||||
<span class="keyword">var</span> range = session.getCommentFoldRange(row, i + match[<span class="number">0</span>].length);
|
||||
range.end.column -= <span class="number">2</span>;
|
||||
<span class="keyword">return</span> range;
|
||||
}</code></pre>
|
||||
<p>Let's say we stumble across the code block <code>hello_world() {</code>. Our range object here becomes:</p>
|
||||
<pre><code class="language-json">{
|
||||
startRow: 0,
|
||||
endRow: 0,
|
||||
startColumn: 0,
|
||||
endColumn: 13
|
||||
}</code></pre>
|
||||
|
||||
<h2>Testing Your Highlighter</h2>
|
||||
|
||||
<p>The best way to test your tokenizer is to see it live, right? To do that, you'll want to modify the <a href="http://ace.ajax.org/build/kitchen-sink.html">live Ace demo</a> to preview your changes. You can find this file in the root Ace directory with the name <em>kitchen-sink.html</em>.</p>
|
||||
<p>The file that defines the behavior for this live demo is defined in <em>demo/kitchen-sink/demo.js</em>. You'll want to add lines to two separate objects:</p>
|
||||
<ol>
|
||||
<li><p><code>modesByName</code> needs a new entry that defines all the rules regarding your new mode. Entries looks like <code>propertyName: [dropdownName, arrayOfExtensions]</code>, where:</p>
|
||||
<ul>
|
||||
<li><code>propertyName</code> is the name of the new language you're highlighting</li>
|
||||
<li><code>dropdownName</code> is an arbitrary string that lists your language in the live demo's <strong>Mode</strong> dropdown menu</li>
|
||||
<li><code>arrayOfExtensions</code> is an array of strings (seperated by <code>|</code>) that defines valid extensions to use for the new language. </li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><p><code>docs</code> also needs a new entry, which defines the location of your sample document showing all the power of your new language. Entries look like <code>filenamePath: modeToUse</code>, where:</p>
|
||||
<ul>
|
||||
<li><code>filenamePath</code> is the path to your example document. This should just be in <em>docs/</em>.</li>
|
||||
<li><code>modeToUse</code> is the same arbitrary string as <code>dropdownName</code></li>
|
||||
</ul>
|
||||
</li>
|
||||
</ol>
|
||||
<p>Once you set this up, you should see be able to witness a live demonstration of your new highlighter.</p>
|
||||
<h3 id='adding-automated-tests'><a class='heading_anchor' href='#adding-automated-tests'><i class='headerLinkIcon'></i></a>Adding Automated Tests</h3>
|
||||
<p>It's also suggested that you go one step further and define some automated tests to test your syntax highlighter. A basic test looks something like this:</p>
|
||||
<pre><code class="language-javascript"><span class="keyword">if</span> (<span class="keyword">typeof</span> process !== <span class="string">"undefined"</span>) {
|
||||
require(<span class="string">"amd-loader"</span>);
|
||||
}
|
||||
|
||||
define(<span class="keyword">function</span>(require, exports, module) {
|
||||
<span class="string">"use strict"</span>;
|
||||
|
||||
<span class="keyword">var</span> MyNewLanguage = require(<span class="string">"./mynewlanguage"</span>).Mode;
|
||||
<span class="keyword">var</span> assert = require(<span class="string">"../test/assertions"</span>);
|
||||
|
||||
module.exports = {
|
||||
|
||||
name: <span class="string">"MyNewLanguage Tokenizer"</span>,
|
||||
|
||||
setUp : <span class="keyword">function</span>() {
|
||||
<span class="keyword">this</span>.tokenizer = <span class="keyword">new</span> MyNewLanguage().getTokenizer();
|
||||
},
|
||||
|
||||
<span class="string">"test: my random description"</span> : <span class="keyword">function</span>() {
|
||||
<span class="keyword">var</span> line = <span class="string">"-12px"</span>;
|
||||
<span class="keyword">var</span> tokens = <span class="keyword">this</span>.tokenizer.getLineTokens(line, <span class="string">"ruleset"</span>).tokens;
|
||||
|
||||
assert.equal(<span class="number">2</span>, tokens.length);
|
||||
assert.equal(<span class="string">"constant.numeric"</span>, tokens[<span class="number">0</span>].type);
|
||||
},
|
||||
|
||||
<span class="string">"test: tokenize more than one thing"</span> : <span class="keyword">function</span>() {
|
||||
<span class="keyword">var</span> tokens = <span class="keyword">this</span>.tokenizer.getLineTokens(<span class="string">"{()}"</span>, <span class="string">"start"</span>).tokens;
|
||||
|
||||
assert.equal(<span class="number">3</span>, tokens.length);
|
||||
assert.equal(<span class="string">"paren.lparen"</span>, tokens[<span class="number">0</span>].type);
|
||||
assert.equal(<span class="string">"text"</span>, tokens[<span class="number">1</span>].type);
|
||||
assert.equal(<span class="string">"paren.rparen"</span>, tokens[<span class="number">2</span>].type);
|
||||
},
|
||||
};
|
||||
|
||||
});
|
||||
|
||||
<span class="keyword">if</span> (<span class="keyword">typeof</span> module !== <span class="string">"undefined"</span> && module === require.main) {
|
||||
require(<span class="string">"asyncjs"</span>).test.testcase(module.exports).exec()
|
||||
}</code></pre>
|
||||
<p>All tests contain a string describing what the test is doing. Then, you define an arbitrary line that you want your tokenizer to check. It's recommended that you test both the number of tokens found, as well as the values of those tokens.</p>
|
||||
<p>Any file ending with the <em>_test.js</em> suffix are automatically run by Ace's <a href="http://travis-ci.org/#!/ajaxorg/ace">Travis CI</a> server.</p>
|
||||
|
||||
</div>
|
||||
<div class="tab-pane fade" id="api">
|
||||
<div class="row centerpiece">
|
||||
<div id="sidebarContainer" class="span3">
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue