style-analyzer: fixing code style inconsistencies with interpretable unsupervised algorithms

Vadim Markovtsev, Hugo Mougard, Waren Long, Egor Bulychev, Konstantin Slavnov - source{d}.

style-analyzer: fixing code ⇥ style␣␣inconsistencies␣with "interpretable"↵'unsupervised' algorithms

Vadim Markovtsev, Hugo Mougard, Waren Long, Egor Bulychev, Konstantin Slavnov
source{d}

Problem

When to help?

  • While you type = IDE
  • While you check = CI
  • While you review = PR
  • Periodically, asynchronously
  • Part of the workflow
  • More time to run the models
  • Nice UI
  • High precision score required
  • Longer feedback loop

Lookout

PR review comments on GitHub

dataset

Classes predicted

space
tabulation
newline
␣+/- spaced indentation increase/decrease
→+/- tabulated indentation increase/decrease
'/" single/double quotes
empty gaps between non-label nodes, NOOP

"White box" is the key

Methodology

Labels predicted

We concatenate common classes together.

No BPE or similar reduction is required.

Annotated code snippet

functionclassesToArray(value){
                if(isArray(value)){returnvalue;}
                 if(typeofvalue===␣"string"␣){
                    returnvalue.match(rnothtml)||[];
                }
                 return[];
             }
        

Augmented token stream

a = b * 2

Machine Learning

Machine Learning

Rules

Rules optimization

a>5 Λ c Λ b>2 Λ d Λ a>10 ⇒ α(merge)
a>10 Λ c Λ b>2 Λ d ⇒ α(redundant)
a>10 Λ c Λ d ⇒ α(feature exclusion)
a>10 Λ c ⇒ α(confidence threshold)

Result

Redundant attributes elimination

Annotated code snippet

functionclassesToArray(value){
                if(isArray(value)){returnvalue;}
                 if(typeofvalue===␣"string"␣){
                    returnvalue.match(rnothtml)||[];
                }
                 return[];
             }
        

Rule example

y = ∅
ŷ = ↲→+
Confidence: 0.975
Support: 3230

Inference

  1. Apply rules
    • Fixes to old code?
    • AST breakage?
    • Identification?
  2. Generate code
    • Indentation?
    • Multiple lines?

Debugging

Lookout comments on GitHub

Evaluation

Precision > Recall

Precision

Prediction Rate

Results

train samples rules precision prediction rate
node 374,298 641 0.965 0.951
jquery 197,072 391 0.972 0.959
axios 21,130 143 0.940 0.951
carlo 5,529 78 0.878 0.931
 
w. average -- -- 0.943 0.947

Enough quality for a product?

Other evaluation is required

Comparison to linters

style-analyzer linters
Precision
Interpretability
Context
Setup
Fixes

But what if the project already uses a linter?

Mining Software Repositories

Clustering

Given codebases X and Y

style_distance(X, Y) - how similar are the formatting styles?

Common formatting rules analysis

Which formatting styles dominate on GitHub?

Summary

Summary

Thank you

bit.ly/2LRkWdW