| Age | Commit message (Collapse) | Author |
|---|
| https://github.com/ruby/prism/commit/09c59a3aa5 |
| Mostly around newlines and line continuation. * percent arrays need special backslash handling in the ast * Fix offset issue for heredocs with many line continuations (used wrong variable as index access) * More refined rules on when to simplify string tokens * Handle line continuations in squiggly heredocs * Correctly dedent squiggly heredocs with interpolation * Consider `':foo:` and `%s[foo]` to not be interpolation https://github.com/ruby/prism/commit/4edfe9d981 |
| Turns out, it was already almost correct. If you disregard \c and \M style escapes, only a single character is allowed to be escaped in a regex so most tests passed already. There was also a mistake where the wrong value was constructed for the ast, this is now fixed. One test fails because of this, but I'm fairly sure it is because of a parser bug. For `/\“/`, the backslash is supposed to be removed because it is a multibyte character. But tbh, I don't entirely understand all the rules. Fixes more than half of the remaining ast differences for rubocop tests https://github.com/ruby/prism/commit/e1c75f304b |
| Also fixes a token incompatibility for the word separator. parser only considers whitespace until the first newline https://github.com/ruby/prism/commit/bd3dd2b62a |
| When the line contains no real newline but contains unescaped ones, then there will be one less entry https://github.com/ruby/prism/commit/4ef093b600 |
| translator This is a followup to #3373, where the implementation was extracted https://github.com/ruby/prism/commit/2637007929 |
| The offset cache contains an entry for each byte so it can't be accessed via the string length. Adds tests for all variants except for this: ``` "fo o" "ba ’" ``` For some reason, this still has the wrong offset. https://github.com/ruby/prism/commit/a651126458 |
| There are a few other locations that should be included in that check. I think the end location must always be present but I left it in to be safe (maybe implicit begin somehow?) https://github.com/ruby/prism/commit/545d07ddc3 |
| `Integer#chr` performs some validation that we don't want/need. Octal escapes can go above 255, where it will then raise trying to convert. `append_as_bytes` actually allows to pass a number, so we can just skip that call. Although, on older rubies of course we still need to handle this in the polyfill. I don't really like using `pack` but don't know of another way to do so. For the utf-8 escapes, this is not an issue. Invalid utf-8 in these is simply a syntax error. https://github.com/ruby/prism/commit/161c606b1f |
| Mostly around newlines and line continuation. * percent arrays need special backslash handling in the ast * Fix offset issue for heredocs with many line continuations (used wrong variable as index access) * More refined rules on when to simplify string tokens * Handle line continuations in squiggly heredocs * Correctly dedent squiggly heredocs with interpolation * Consider `':foo:` and `%s[foo]` to not be interpolation https://github.com/ruby/prism/commit/4edfe9d981 |
| I want to add new node types to the parser translator, for example `itblock`. The bulk of the work is already done by prism itself. In the `parser` builder, this would be a 5-line change at most but we don't control that here. Instead, we can add our own builder and either overwrite the few methods we need, or just inline the complete builder. I'm not sure yet which would be better. `rubocop-ast` uses its own builder for `parser`. For this to correctly work, it must explicitly choose to extend the prism builder and use it, same as it currently chooses to use a different parser when prism is used. I'd like to enforce that the builder for prism extends its custom one since it will lead to some pretty weird issues otherwise. But first, I'd like to change `rubocop-ast` to make use of this. https://github.com/ruby/prism/commit/b080e608a8 |
| 1. The string starts out as binary 2. `ち` is appended, forcing it back into utf-8 3. Some invalid byte sequences are tried to append > incompatible character encodings: UTF-8 and BINARY (ASCII-8BIT) This makes use of my wish to use `append_as_bytes`. Unfortunatly that method is rather new so it needs a fallback https://github.com/ruby/prism/commit/e31e94a775 |
| Previously, the comparison code would loop through segments up to the longest of the two versions being compared. However, this is inefficient because once one version has more segments than the other we can do a lot less work. This commit optimizes the differing segment length case by specializing the logic once the iteration has passed the shorter of the two segment lengths. At this point we only need to continue looking at the longer version's segment, and we know that any String encountered means the version is less than (pre), and any non-zero Integer means the version is greater than. Benchmark: ``` { first: [Gem::Version.new("1.2.3"), Gem::Version.new("2.2.3")], second: [Gem::Version.new("1.2.3"), Gem::Version.new("1.3.3")], third: [Gem::Version.new("1.2.3"), Gem::Version.new("1.2.4")], length: [Gem::Version.new("1.2.3"), Gem::Version.new("1.2.3.4")], left_s_second: [Gem::Version.new("1.a.3"), Gem::Version.new("1.2.3")], left_s_third: [Gem::Version.new("1.2.a"), Gem::Version.new("1.2.3")], right_s_second: [Gem::Version.new("1.2.3"), Gem::Version.new("1.a.3")], right_s_third: [Gem::Version.new("1.2.3"), Gem::Version.new("1.2.a")], left_s_length: [Gem::Version.new("8.0.1.pre"), Gem::Version.new("8.0.1")], right_s_length: [Gem::Version.new("8.0.1"), Gem::Version.new("8.0.1.pre")], both_s: [Gem::Version.new("8.0.2.pre1"), Gem::Version.new("8.0.2.pre2")], }.each do |name, v| puts "== #{name} ==" raise name unless v[0].fast_comp(v[1]) == (v[0] <=> v[1]) Benchmark.ips do |x| x.report("fast") { v[0].fast_comp(v[1]) } x.report("original") { v[0] <=> v[1] } x.compare!(order: :baseline) end end == first == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 208.555k i/100ms original 199.789k i/100ms Calculating ------------------------------------- fast 2.075M (± 6.0%) i/s (481.93 ns/i) - 10.428M in 5.055818s original 2.045M (± 3.9%) i/s (488.94 ns/i) - 10.389M in 5.090034s Comparison: fast: 2075002.8 i/s original: 2045227.4 i/s - same-ish: difference falls within error == second == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 192.395k i/100ms original 183.000k i/100ms Calculating ------------------------------------- fast 1.892M (± 3.8%) i/s (528.62 ns/i) - 9.620M in 5.094104s original 1.824M (± 3.5%) i/s (548.11 ns/i) - 9.150M in 5.023163s Comparison: fast: 1891722.2 i/s original: 1824435.3 i/s - same-ish: difference falls within error == third == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 172.788k i/100ms original 162.934k i/100ms Calculating ------------------------------------- fast 1.719M (± 9.0%) i/s (581.72 ns/i) - 8.467M in 5.025861s original 1.638M (± 3.6%) i/s (610.36 ns/i) - 8.310M in 5.080344s Comparison: fast: 1719042.9 i/s original: 1638389.6 i/s - same-ish: difference falls within error == length == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 191.741k i/100ms original 155.952k i/100ms Calculating ------------------------------------- fast 1.920M (± 3.9%) i/s (520.74 ns/i) - 9.587M in 5.002328s original 1.576M (± 6.2%) i/s (634.42 ns/i) - 7.954M in 5.072507s Comparison: fast: 1920362.1 i/s original: 1576240.9 i/s - 1.22x slower == left_s_second == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 176.441k i/100ms original 164.879k i/100ms Calculating ------------------------------------- fast 1.609M (± 7.3%) i/s (621.51 ns/i) - 8.116M in 5.083414s original 1.620M (± 8.3%) i/s (617.43 ns/i) - 8.079M in 5.028525s Comparison: fast: 1608994.8 i/s original: 1619606.5 i/s - same-ish: difference falls within error == left_s_third == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 160.562k i/100ms original 152.799k i/100ms Calculating ------------------------------------- fast 1.591M (± 3.6%) i/s (628.40 ns/i) - 8.028M in 5.052029s original 1.528M (± 3.6%) i/s (654.31 ns/i) - 7.640M in 5.007526s Comparison: fast: 1591334.1 i/s original: 1528320.6 i/s - same-ish: difference falls within error == right_s_second == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 135.938k i/100ms original 132.907k i/100ms Calculating ------------------------------------- fast 1.367M (± 1.2%) i/s (731.77 ns/i) - 6.933M in 5.074030s original 1.320M (± 2.4%) i/s (757.35 ns/i) - 6.645M in 5.036155s Comparison: fast: 1366548.7 i/s original: 1320386.4 i/s - same-ish: difference falls within error == right_s_third == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 129.971k i/100ms original 123.802k i/100ms Calculating ------------------------------------- fast 1.273M (± 4.1%) i/s (785.25 ns/i) - 6.369M in 5.011805s original 1.215M (± 1.8%) i/s (823.04 ns/i) - 6.190M in 5.096330s Comparison: fast: 1273487.0 i/s original: 1215002.9 i/s - same-ish: difference falls within error == left_s_length == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 211.093k i/100ms original 155.784k i/100ms Calculating ------------------------------------- fast 2.120M (± 1.9%) i/s (471.63 ns/i) - 10.766M in 5.079336s original 1.565M (± 6.7%) i/s (638.87 ns/i) - 7.789M in 5.007522s Comparison: fast: 2120296.1 i/s original: 1565258.0 i/s - 1.35x slower == right_s_length == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 213.977k i/100ms original 142.990k i/100ms Calculating ------------------------------------- fast 2.154M (± 1.3%) i/s (464.15 ns/i) - 10.913M in 5.066124s original 1.446M (± 1.8%) i/s (691.75 ns/i) - 7.292M in 5.046172s Comparison: fast: 2154455.3 i/s original: 1445607.9 i/s - 1.49x slower == both_s == ruby 3.4.2 (2025-02-15 revision https://github.com/rubygems/rubygems/commit/d2930f8e7a) +PRISM [arm64-darwin23] Warming up -------------------------------------- fast 154.903k i/100ms original 131.011k i/100ms Calculating ------------------------------------- fast 1.515M (± 4.0%) i/s (659.97 ns/i) - 7.590M in 5.019890s original 1.280M (± 5.3%) i/s (781.28 ns/i) - 6.420M in 5.035387s Comparison: fast: 1515223.3 i/s original: 1279957.8 i/s - 1.18x slower ``` https://github.com/rubygems/rubygems/commit/7195e77152 |
| Enum array may be the list of pairs of key and value. Check if only key is completable, not pair. Fix https://github.com/ruby/optparse/pull/93 Fix https://github.com/ruby/optparse/pull/94 https://github.com/ruby/optparse/commit/a8d0ba8dac |
| https://github.com/ruby/prism/commit/422d5c4c64 |
| Avoids an array allocation which matters more and more the larger the file is. I have it at 14% of runtime. https://github.com/ruby/prism/commit/f65b90f27d |
| I see `Array.include?` as 2.4% runtime. Probably because of `LPAREN_CONVERSION_TOKEN_TYPES` but the others will be faster as well. Also remove some inline array checks. They are specifically optimized in Ruby since 3.4, but for now prism is for >= 2.7 https://github.com/ruby/prism/commit/ca9500a3fc |
| Temoprary backwards-compat code so that current users don't break. Eventually the Translation::Parser initializer should asser that the correct class is passed in. https://github.com/ruby/prism/commit/66b0162b35 |
| In a large application I profiled allocations while running `bundle update` and found that this method was ~60% of allocations while resolving (and Candidate#<=> is almost half of the total runtime). This commit removes the array allocation in Candidate#<=> (and similar methods since the implementations are so simple). The array is always the same two elements so they can just be compared directly. https://github.com/rubygems/rubygems/commit/6a7c411ba7 |
| https://github.com/ruby/prism/commit/56eaf53732 |
| https://github.com/ruby/prism/commit/10e5431b38 |
| The with and without flags accepts both comma and space separated values. https://github.com/rubygems/rubygems/commit/b6149f61e3 |
| If a custom rubygems source URI is long enough, Bundler may end up raising an `ENAMETOOLONG` error and crash. This commit fixes the problem by trimming the cache slug size to fit usual OS requirements. https://github.com/rubygems/rubygems/commit/df40ff1e14 Co-authored-by: mbclu <mbclu@users.noreply.github.com> Co-authored-by: martinemde <martinemde@users.noreply.github.com> |
| ## Summary `itblock` node is added to support the `it` block parameter syntax introduced in Ruby 3.4. ```console $ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \ p Prism::Translation::Parser34.new.tokenize(buffer)[0]' s(:itblock, s(:send, nil, :proc), :it, s(:lvar, :it)) ``` This node design is similar to the `numblock` node, which was introduced for the numbered parameter syntax in Ruby 2.7. ``` $ ruby -Ilib -rprism -rprism/translation/parser34 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { _1 }"; \ p Prism::Translation::Parser34.new.tokenize(buffer)[0]' s(:numblock, s(:send, nil, :proc), 1, s(:lvar, :_1)) ``` The difference is that while numbered parameters can have multiple parameters, the `it` block parameter syntax allows only a single parameter. In Ruby 3.3, the conventional node prior to the `it` block parameter syntax is returned. ```console $ ruby -Ilib -rprism -rprism/translation/parser33 -e 'buffer = Parser::Source::Buffer.new("path"); buffer.source = "proc { it }"; \ p Prism::Translation::Parser33.new.tokenize(buffer)[0]' s(:block, s(:send, nil, :proc), s(:args), s(:send, nil, :it)) ``` ## Development Note The Parser gem does not yet support the `it` block parameter syntax. This is the first case where Prism's node design precedes that of the Parser gem. When implementing https://github.com/whitequark/parser/issues/962, this node design will need to be taken into consideration. https://github.com/ruby/prism/commit/c141e1420a |
| https://github.com/ruby/optparse/commit/f4d64b0b17 |
| https://github.com/ruby/optparse/commit/83e8c23d68 |
| If the original value of LESS ends with an option starting with "--", simply appending "Fe" would result in an invalid option string. https://github.com/ruby/optparse/commit/30571f91d3 |
| In mock testing for stdout, `StringIO.new` is sometimes used to redirect the output. In such cases, the assignment is done with `$stdout = StringIO.new`, not the constant `STDOUT`. e.g., https://github.com/rubocop/rubocop/blob/v1.71.1/lib/rubocop/rspec/shared_contexts.rb#L154-L164 After assigning `StringIO.new`, `$stdout.tty?` returns `false`, allowing the standard output destination to be switched during test execution. ```ruby STDOUT.tty? # => true StringIO.new.tty? # => false ``` However, since `STDOUT.tty?` returns `true`, a failure occurred in environments where the environment variables `RUBY_PAGER` or `PAGER` are set. e.g., https://github.com/rubocop/rubocop/pull/13784 To address this, `STDOUT` has been updated to `$stdout` so that the result of `tty?` can be flexibly overridden. A potential concern is that `$stdout`, unlike `STDOUT`, does not always represent the standard output at the time the Ruby process started. However, no concrete examples of issues related to this have been identified. `STDOUT.tty?` is the logic of optparse introduced in https://github.com/ruby/optparse/pull/70. This PR replaces `STDOUT` with `$stdout` throughout, based on the assumption that `$stdout` is sufficient for use with optparse. https://github.com/ruby/optparse/commit/262cf6f9ac |
| Fix https://github.com/ruby/optparse/pull/80 https://github.com/ruby/optparse/commit/050a87d029 |
| https://github.com/ruby/optparse/commit/71e2b31824 |
| https://github.com/ruby/optparse/commit/d7dec6808f |
| backqoutes https://github.com/ruby/optparse/commit/5e71a70cb5 |
| https://github.com/rubygems/rubygems/commit/369f9b9311 Notes: Merged: https://github.com/ruby/ruby/pull/12890 |
| only the '.git' directory is present. This recovers cases where a git-sourced install can be left in a partially installed state. https://github.com/rubygems/rubygems/commit/d132b7008d Notes: Merged: https://github.com/ruby/ruby/pull/12890 |
| https://github.com/rubygems/rubygems/commit/ba5a62fd04 Notes: Merged: https://github.com/ruby/ruby/pull/12890 |
| https://github.com/rubygems/rubygems/commit/9691097036 Notes: Merged: https://github.com/ruby/ruby/pull/12890 |
| Command line arguments are strings, convert enum list elements to strings to match. https://github.com/ruby/optparse/commit/c5ec052efc |
| https://github.com/ruby/optparse/commit/a3f1029815 |
| Make `rdoc .` and `rake rdoc` consistent. https://github.com/ruby/optparse/commit/61b4ea0704 |
| https://github.com/rubygems/rubygems/commit/3b4934fb69 Notes: Merged: https://github.com/ruby/ruby/pull/12840 |
| option https://github.com/rubygems/rubygems/commit/c258e45b44 Notes: Merged: https://github.com/ruby/ruby/pull/12840 |
| https://github.com/rubygems/rubygems/commit/591d2c0503 Notes: Merged: https://github.com/ruby/ruby/pull/12840 |
| https://github.com/rubygems/rubygems/commit/176dc7421c Notes: Merged: https://github.com/ruby/ruby/pull/12840 |
| https://github.com/rubygems/rubygems/commit/fafb9ae090 |
| https://github.com/ruby/cgi/commit/ab84b7fe66 |
| https://github.com/ruby/cgi/commit/8e6fb1041b |
| It has been over a year since the release, so let's stop MD5ing everything https://github.com/rubygems/rubygems/commit/29ef4ca30b |
| If we fail to write the lockfile, give a better error. https://github.com/rubygems/rubygems/commit/81a08d6eda |
| Currently, some warning messages don't contain a `URI` like the following. ```ruby warning: URI::ABS_URI is obsolete. Use RFC2396_PARSER.regexp[:ABS_URI] explicitly. ``` But, without `URI` prefix, the suggested value doesn't work. So I think we should use a fully qualified name to avoid confusion. https://github.com/ruby/uri/commit/428eb10e44 |
| This was removed by #9. https://github.com/ruby/uri/commit/fec924238f |
