-
Notifications
You must be signed in to change notification settings - Fork 814
Add HLL lexer #2172
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add HLL lexer #2172
Changes from 9 commits
99fff6e
6c030e5
9b34df3
ebeee5e
92655e6
beafe20
a088b7f
f459ae6
56b539f
fbfe7b6
395bc9c
ce7d888
6434243
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| /* The pigeon holes principles | ||
| Taken from the specification */ | ||
| constants: | ||
| int NOF_PIGEONS := 10; | ||
| int NOF_HOLES := NOF_PIGEONS - 1; | ||
|
|
||
| inputs: | ||
| bool P[NOF_PIGEONS, NOF_HOLES]; | ||
|
|
||
| definitions: | ||
| // For each hole there is at most one pigeon | ||
| a := ALL i:[0, NOF_HOLES-1], j:[0, NOF_PIGEONS-1] ALL k:[j+1, NOF_PIGEONS-1] | ||
| (P[j, i] -> ~P[k, i]); | ||
|
|
||
| // For each pigeon there is at least one hole | ||
| b := ALL i:[0, NOF_PIGEONS-1] SOME j:[0, NOF_HOLES-1] (P[i,j]); | ||
|
|
||
| proof obligations: | ||
| ~(a & b); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| module Rouge | ||
| module Lexers | ||
| class HLL < RegexLexer | ||
| title 'HLL' | ||
| desc 'High Level Language' | ||
| tag 'hll' | ||
| filenames '*.hll' | ||
| mimetypes 'text/x-hll' | ||
|
|
||
| state :comment do | ||
| rule %r(///.*?$), Comment::Doc | ||
| rule %r(//[^/].*?$), Comment::Single | ||
| rule %r(/\*[^\*].*?\*/)m, Comment::Multiline | ||
| rule %r(/\*\*.*?\*/)m, Comment::Doc | ||
| end | ||
|
|
||
| state :root do | ||
| rule %r(\s+), Text::Whitespace | ||
| rule %r/\w::/, Name::Namespace | ||
| rule %r/"[^"]*"/, Literal::String | ||
| rule %r/'[^']*'/, Literal::String | ||
| mixin :comment | ||
| rule %r([(){}\[\];:,|!^]+), Punctuation | ||
| rule %r(:=), Punctuation | ||
| rule %r(\d+), Literal::Number | ||
| rule %r(\w+) do |m| | ||
| if self.class.keywords.include?(m[0]) | ||
| token Keyword | ||
| elsif self.class.type_keywords.include?(m[0]) | ||
| token Keyword::Type | ||
| else | ||
| token Name | ||
| end | ||
| end | ||
| rule %r([~&#+\-><*=]), Operator | ||
| end | ||
|
|
||
| def self.type_keywords | ||
| @type_keywords ||= Set.new %w(bool int) | ||
| end | ||
| # Keywords as indicated in | ||
| # https://hal.science/hal-03356342v1/ | ||
| # but without types (which are in the type_keywords value) | ||
| def self.keywords | ||
| @keywords ||= Set.new %w( | ||
| ALL | ||
| bin2s | ||
| bin2u | ||
| Blocks | ||
| blocks | ||
| cast | ||
| CONJ | ||
| constants | ||
| Constants | ||
| constraints | ||
| Constraints | ||
| declarations | ||
| Declarations | ||
| definitions | ||
| Definitions | ||
| DISJ | ||
| elif | ||
| else | ||
| false | ||
| FALSE | ||
| False | ||
| I | ||
| if | ||
| inputs | ||
| Inputs | ||
| lambda | ||
| namespaces | ||
| Namespaces | ||
| obligations | ||
| Obligations | ||
| outputs | ||
| Outputs | ||
| population_count_eq | ||
| population_count_gt | ||
| population_count_lt | ||
| pre | ||
| PRE | ||
| PROD | ||
| proof | ||
| Proof | ||
| s2bin | ||
| SELECT | ||
| signed | ||
| SOME | ||
| sort | ||
| struct | ||
| SUM | ||
| then | ||
| true | ||
| True | ||
| TRUE | ||
| tuple | ||
| types | ||
| Types | ||
| u2bin | ||
| unsigned | ||
| with | ||
| X) | ||
| end | ||
| end | ||
| end | ||
| end | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| describe Rouge::Lexers::HLL do | ||
| let(:subject) { Rouge::Lexers::HLL.new } | ||
|
|
||
| describe 'guessing' do | ||
| include Support::Guessing | ||
|
|
||
| it "guesses by filename" do | ||
| assert_guess :filename => "foo.hll" | ||
| end | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| /* The pigeon holes principles | ||
| Taken from the specification */ | ||
| constants: | ||
| int NOF_PIGEONS := 10; | ||
| int NOF_HOLES := NOF_PIGEONS - 1; | ||
|
|
||
| inputs: | ||
| bool P[NOF_PIGEONS, NOF_HOLES]; | ||
|
|
||
| definitions: | ||
| // For each hole there is at most one pigeon | ||
| a := ALL i:[0, NOF_HOLES-1], j:[0, NOF_PIGEONS-1] ALL k:[j+1, NOF_PIGEONS-1] | ||
| (P[j, i] -> ~P[k, i]); | ||
|
|
||
| // For each pigeon there is at least one hole | ||
| b := ALL i:[0, NOF_PIGEONS-1] SOME j:[0, NOF_HOLES-1] (P[i,j]); | ||
|
|
||
| proof obligations: | ||
| ~(a & b); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If it's possible, it's best to really stress-test the lexer here, rather than pasting the same example from the demo.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I created a new stress test that should cover most common syntax constructions. Don't hesitate to tell me if you prefer an even more complete coverage. |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm having trouble finding this in the docs - are these keywords case insensitive, or are only the cases listed here (like
true,True,TRUE) valid? If it's case-insensitive, it's probably better to upcase or downcase before checking inclusion in this set.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only the cases listed here are valid. In https://hal.science/hal-03356342v1/, you can see in Appendix B the correct spellings. Therefore, as you mention, I don't think it's necessary to downcase before checking.