Ash: A Syn­tac­tic Ed­i­tor

What is this mad­ness?!

Full dis­clo­sure, I'm an able user of Vi. I love Vi for two rea­sons: It has modes, sorry Larry, and it en­able nav­i­ga­tion and edit­ing us­ing text ob­jects. In Vi you don't edit text, you edit para­graphs, sen­tences, and words. This means that you can "talk" to your ed­i­tor; "please delete the next 3 words" or "copy the para­graph". When you learn to talk the lan­guage, there is clear com­mu­ni­ca­tion be­tween you and the ed­i­tor. Thought there is a prob­lem. The prob­lem is that code is not sen­tences and words, it's func­tions, ex­pres­sions and de­c­la­ra­tions. So even thought, you are able to talk to your ed­i­tor, it has no idea what it's do­ing. Of cause, many peo­ple have tried to ban­dage the sys­tem, and the ed­i­tor might get to know about a func­tion in one pro­gram­ming lan­guage, and that might work well in some cases, but deep down it's noth­ing but char­ac­ters in a string.

This in­ter­nal rep­re­sen­ta­tion is why we can't have nice things. If plug-in builders has to rein­vent the wheel, by more or less try­ing to learn your ed­i­tor to read code every time they want to add a new fea­ture, then it's not so strange that each plug-in can only be so ad­vanced, and that the plug-ins does not nec­es­sar­ily talk to­gether.

Ash tries to fix this. Ash is a syn­tac­tic ed­i­tor. This means that the in­ner rep­re­sen­ta­tion of a file or buffer is not a string of char­ac­ters, but a syn­tax tree. For ex­am­ple, the ex­pres­sion you saw in the ed­i­tor as you opened the page, were 1+23+45. By the way the box in the be­gin­ning of this blog, is the full ed­i­tor, feel free to play around with it. That ex­pres­sion is rep­re­sented like a tree in Ash.

You can see this for your­self by writ­ing :display debug; to get back write :display simple.

This dif­fer­ent in­ter­nal rep­re­sen­ta­tion al­lows us to do a lot of cool things, that we can't, or are at least hard to, do in a nor­mal tex­tural ed­i­tor:

• Syn­tac­tic move­ments — Move around in the file in log­i­cal por­tions. Like with Vi, but in­stead of talk­ing in words and sen­tences, you can now say func­tion and de­c­la­ra­tion.
• Syn­tac­tic edit­ing — Edit­ing is just re­mov­ing or adding things to a tree. But the three has to be well formed. The tree can make qual­i­fied guessed de­pend­ing on where you are in the tree. The well-formed­ness also mean that you have to say good­bye to your last syn­tax er­ror. This ed­i­tor sim­ply doesn't al­low you to make syn­tax er­rors be­cause they can't fit in the tree.
• Ad­vanced pretty print­ing — Be­cause it is a tree, there is re­ally no rea­son that you should be lim­ited to view the code as text. Print things as it makes most sense to you.

Hasn't it been done be­fore?

To my knowl­edge, no. But at­tempts at adding syn­tac­tic edit­ing to other ed­i­tors has been made. Most odi­ous ex­am­ples is some­thing like snip­pets in­tro­duced in Text­Mate, or refac­tor­ing like in Eclipse. But this is more in­ser­tions into text than true syn­tax edit­ing. While ex­pand­ing a snip­pet you tem­po­rally get the abil­ity to do re­ally cool things, like cre­at­ing classes with a few key strokes or au­to­mat­i­cally cre­at­ing a get­ter and set­ter while mak­ing a new at­tribute. But as soon as you ac­cept the change the code be­comes text, and you have to edit it man­u­ally again.

More in­spi­ra­tional for me was two modes to modes for Emacs, paredit-mode and struc­tured-haskell-mode. Paredit is a mi­nor mode, that en­ables the user to quickly write Lisp. Struc­tured Haskell does the same for, you guessed it, Haskell. They uses spe­cial com­mands to write new ex­pres­sions and move around in the source. There is ac­tu­ally a nice YouTube video show­ing paredit in all its glory. Even though the two modes are cool, there are two ma­jor draw­backs: They only work on Lisp and Haskell, and it is way to easy to break out of the mode and break the syn­tax. The last thing is a clear side ef­fect of be­ing build on top of a, granted pow­er­ful, tex­tural ed­i­tor.

I have also been in­spired by Ohm, which is a parser ex­pres­sion gram­mar cre­ator, and it in­spired me to build my in­ner rep­re­sen­ta­tion.

Why would you do such a thing?

I set out with a goal to show that syn­tac­tic edit­ing not only is pos­si­ble but also bet­ter than do­ing tex­tural edit­ing. I re­fraced this into three high level ques­tions.

1. It is pos­si­ble, to cre­ate a min­i­mal vi­able pro­to­type that ed­its syn­tax trees in­stead of text?
2. How do we nav­i­gate in such a syn­tac­tic ed­i­tor?
3. Are ed­its more id­iomatic, or is it more con­fus­ing to work in the syn­tac­tic ed­i­tor than work­ing in an nor­mal ed­i­tor?

I bet you didn't do it alone!

No, I had a lot of help. The pro­ject was com­pleted as part of CS 237B spring 2016, taught at UCLA. The pro­ject, were de­vel­oped over it­er­a­tions with feed­back and valu­able ad­vice from my class­mates and the in­struc­tor, Alex Warth.

What did you do?!

First of all I cre­ated a frame­work that sup­ports a syn­tac­tic ed­i­tor. It is a lot of work to write a parser and find a gen­eral way of work­ing with syn­tax trees, but more about that in the next sec­tion. Let's look at some of the cool things you can do with the ed­i­tor first.

Move­ment

The most es­sen­tial part of an ed­i­tor is strangely enough not edit­ing, it is move­ment. If you can't move around, you can't edit. You move around with Shift+h, Shift+j, Shift+k and Shift+l, yes, I did say I use Vi. Be­cause we work in a tree, move­ment is now some­what dif­fer­ent than in a text ed­i­tor:

• Shift+h moves to the pre­vi­ous sib­ling.
• Shift+j moves to the first child
• Shift+k moves to the par­ent.
• Shift+l moves to the next sib­ling.

In a syn­tac­tic ed­i­tor, there is no such thing as a cur­sor, in­stead there is a fo­cus on a spe­cific node in the syn­tax tree. This first ex­am­ple shows the fo­cus be­ing moved around with the com­mands.

Hover to see the move­ment J-L-H-K.

Even though that kind of move­ment makes sense for the com­puter, it does not make sense for the user. Nav­i­gat­ing through list of num­bers quickly be­comes te­dious work. In this ex­am­ple the user want to se­lect the 6th digit in the num­ber.

Se­lect­ing the 6th el­e­ment is hard work; J-L-J-L-....

To counter this, I in­tro­duce as spe­cial kind of move­ment. I call it smart move­ment. The pur­pose of smart move­ment is to re­move a spe­cial kind of repet­i­tive move­ment. This move­ment hap­pens when you trans­verse lists. In syn­tax trees there ex­ist two kinds of lists. Left-re­cur­sive lists, where the left most child can be of the same kind as the el­e­ment or right-re­cur­sive lists where the right most child can be of the same kind. Since lists are a re­oc­cur­ring el­e­ment in syn­tax trees I have cre­ated a new set of smart move­ments.

• h moves to the pre­vi­ous list sib­ling.
• j moves to the first list child
• k moves to the list par­ent.
• l moves to the next list sib­ling.

If the el­e­ment that is in fo­cus is not a list the move­ment works as nor­mal. But if it is a list it works like this:

Se­lect­ing the 6th el­e­ment is easy; j-l-l-l-....

Or like this, if the list is left re­cur­sive:

Smart move­ment also works for right re­cur­sive lists; j-l-l-l-....

And this is every­thing that you need to move around in a syn­tax tree. Well, search­ing might be nice, but you can't have every­thing in life.

Edit­ing

The next most im­por­tant in thing in an ed­i­tor is edit­ing. Edit­ing in a syn­tax tree is a lot dif­fer­ent than in a tex­tural ed­i­tor. The first ac­tion that you will learn is the change com­mand. Change takes the el­e­ment in fo­cus and re­place it with some­thing that you de­cide.

To start a change ac­tion, press c. Type a lit­tle, and then end the ac­tion by press­ing en­ter.

First baby steps, us­ing change.

No­tice that the change ac­tion is smart; not only does it au­to­mat­i­cally de­tect that the sec­ond op­er­ant of + is not pre­sent and adds a hole "?", but it also fig­ures out that the nu­mer­a­tor of the frac­tion can not be an ad­di­tion with­out sur­round­ing paren­the­sis. On a side node if you press ↓, the ed­i­tor will con­tinue to pro­duce sug­ges­tions. This might be re­ally use­full if you are not fa­mil­iar with the lan­guage and want help with the syn­tax. Press­ing ↑ will get you back again.

Let's talk a lit­tle about holes. Real syn­tax trees can not con­tain bro­ken pieces of syn­tax, but it is in­her­ently hu­man not to think very struc­tured — or the very least not in syn­tax trees. Peo­ple want to do things in an­other or­der, or with an­other goal in mind. So to al­low for that, Ash has holes. Holes are like Java's null. A hole can sub­sti­tute any piece of valid syn­tax. Sadly a hole also rep­re­sent a bro­ken syn­tax tree so the ed­i­tor will try to do any­thing in its power to re­move the holes. See what hap­pens when we try to leave a hole in ex­pres­sion "23 +".

Holes are strange.

As you can see, when the ac­tion is ended, the tool au­to­mat­i­cally jumps to first hole it can find, and starts a new change ac­tion. But, what hap­pened in the last part?. Well you can in­sert holes while typ­ing. Holes can be rep­re­sented dif­fer­ently in dif­fer­ent lan­guage, but the de­fault is "?". This is a pow­er­ful fea­ture be­cause it means that you can type in par­tial syn­tax. This be­comes more im­por­tant in an minute. But first go to the ed­i­tor and try to type "42+?/13+?+23", press en­ter and try to mea­sure how fast you can fill out the holes with num­bers. Now think about how long this op­er­a­tion would have taken you in a tex­tural ed­i­tor.

So far we can only change parts of the code com­pletely. Even though that is nice, when edit­ing larger pro­grams you might not want to throw away all, or a part of the code, just be­cause you want to add some­thing. Two ac­tions has been cre­ated just for that; ap­pend and in­sert. Ap­pend a fills the first valid hole in the new tree with the old fo­cused ex­pres­sion — in­sert i fills the last.

In­sert "?+?"	Ap­pend "?+?"

Warn­ing, this ac­tion do not have au­to­matic paren­the­sis in­ser­tion, or work well with lists... sorry.

The last thing that is es­sen­tial to edit­ing, is to re­move all the things bad code you have just writ­ten. There is two ways to delete el­e­ments in ash.

• d deletes an el­e­ment. If there is only two el­e­ment in the par­ent of the el­e­ment. That el­e­ment will be up­graded to the new main el­e­ments. Else the par­ent will also be deleted.Warn­ing, slightly buggy.
• x ex­tract an el­e­ment. This is the op­po­site of delete. It will keep the cur­rent el­e­ment, and re­place it with the par­ent. It is pos­si­ble to de­cide be­tween to what layer that the el­e­ment should be ex­tracted by press­ing ↓.

Mul­ti­ple buffers and lan­guages

Ash also sup­ports mul­ti­ple buffers, though there is no way for you to see that. To cre­ate a new buffer type ":new <language>". Right now there is two lan­guages to play around with. math as you are al­ready fa­mil­iar with is a sim­ple lan­guage that can do arith­metic. f is a more com­pli­cated func­tional lan­guage, that is al­most, but not quite, en­tirely un­like OCaml. Both of these lan­guage has been more or less stolen from CS 237A taught at UCLA or from Ohm.

Pretty print­ing

Each lan­guage comes with a some build-in pretty print­ers. For the lan­guage math there are three pretty print­ers. Each of them can be ac­cessed by us­ing :display <printer>. The im­ages be­low are pic­tures of the same syn­tax tree be­ing printed with the dif­fer­ent print­ers.

debug	simple	pretty

The two pretty print­ers debug and simple are lan­guage ag­nos­tic ways of dis­play­ing syn­tax trees. debug is the dis­play clos­est to the rep­re­sen­ta­tion of the in­ter­nal syn­tax tree. The num­bers at each node name is the id of the node. simple is just a print of all the to­kens in the tree in or­der.

The most in­ter­est­ing of the three is the pretty printer that ac­cu­rately de­picts the math as math and not as a piece of text. Nav­i­ga­tion is a lot harder in a pretty printed tree, as up an down has a new mean­ing. Play around with it and see what you think.

Sav­ing and load­ing files

Let me an­swer that with a ques­tion; is that re­ally im­por­tant?

How is this pos­si­ble?!

The en­tire thing is writ­ten in elm, ex­cept for some setup code in JavaScript. The code is avail­able at github. Elm is a very cool func­tional pro­gram­ing lan­guage for front-end web de­vel­op­ers, and with­out it I don't think that this pro­ject would have been pos­si­ble. Its pure­ness and type checker catches most of my mis­takes while I pro­gramed. Also this knowl­edge made me more con­fi­dent that when I made rapid changes the ed­i­tor still works.

There went a lot of work into get­ting the ed­i­tor to work as an ed­i­tor, but that is not re­ally in­ter­est­ing or unique for a syn­tac­tic ed­i­tor.

The gram­mar

The first im­por­tant in­tro­duc­tion was the Gram­mar, a mod­ule and a em­bed­ded DSL that en­ables users to write very sim­ple gram­mar in Elm. The gram­mar sys­tem is very in­spired by Ohm which can rep­re­sent pars­ing ex­pres­sion gram­mars, but ash build-in gram­mar is much less pow­er­ful, be­cause it is miss­ing the ex­pres­sions. Be­low is some of the math lan­guage, spec­i­fied in the em­bed­ded gram­mar DSL.

grammar = 
  Grammar.grammar "?"
    [ ( "Exp"
      , rule 
        [ [ Ref "AddExp" ] 
        ]
      )
    , ( "AddExp"
      , rule 
        [ [ Ref "AddExp", Lex "+", Ref "MulExp" ] -- plus
        , [ Ref "AddExp", Lex "-", Ref "MulExp" ] -- minus
        , [ Ref "MulExp" ]
        ]
      )
    , ( "MulExp"
      , rule
        [ [ Ref "MulExp", Lex "*", Ref "ExpExp" ] -- times
        , [ Ref "MulExp", Lex "/", Ref "ExpExp" ] -- divide
        , [ Ref "ExpExp" ]
        ]
      )
    , ...
    ]

No­tice the "?" in sec­ond line. That is how you spec­ify how the gram­mar should parse holes. The holes are added to all the end of all ex­pres­sions in all of the rules.

The syn­tax tree

Syn­tax tree is the prod­uct of pars­ing a string with a gram­mar. The syn­tax tree is generic, and is should be able to hold all pos­si­ble trees. Here is the type de­c­la­ra­tion. It might not mean a lot to you, but the poly­mor­phism of SNode en­abled a lot of cool tree col­lec­tors.

type alias SNode a
  = { kind  : SyntaxKind
    , terms : List a
    , size  : Int
    }

-- Recusive type to please the type checker 
type SubTree = SubTree (SNode SubTree)

type alias SyntaxTree = SNode SubTree

SNode con­tains tree things a kind, which is a ref­er­ence to an ex­pres­sion in a gram­mar some­where, a list of sub nodes, called terms, and a size. The size is used to give each node an iden­ti­fier. Id's are cru­cial, when we want to ref­er­ence a cer­tain node. In­stead of giv­ing each node a sta­tic id, the id is cal­cu­lated in re­la­tion to where in the tree it is. The id's are as­signed in a depth first man­ner. The prob­lem with this dy­namic iden­ti­fier as­sign­ment is that it is not very ef­fi­cient. To find the iden­ti­fier of a node the en­tire tree has to be trans­versed to find a node with a spe­cific id. To fix this I added the size to the node. The size is the to­tal of nodes in the node in­clud­ing it­self. Us­ing this num­ber it is pos­si­ble to di­rect the search. Sadly be­cause of the iden­ti­fier sys­tem it is nec­es­sary to in­tro­duce tree col­lec­tors. A tree it­er­a­tor could it­er­ate the tree and would stop as soon as any value were re­turned.

type alias TreeIterator a = Int -> SyntaxTree -> Maybe a

A tree col­lec­tor would col­lect there re­sults of call­ing the it­er­a­tor re­cur­sively and re­place the nodes in the terms ar­ray with the re­sults.

type alias TreeCollector a = Int -> SNode a -> a

One thing that turned out to be ex­cep­tion­ally hard was to make sure that af­ter a tree was changed, code point­ing to nodes in the tree was able to up­date the ref­er­ences to some­thing that made sense. The so­lu­tion was that, when a tree was up­dated, it also pro­duced a func­tion that could up­date any valid iden­ti­fier to a new iden­ti­fier point­ing to the same node or at the near­est par­ent node. Gen­er­at­ing this func­tion au­to­mat­i­cally for any up­date turned out to be hard and re­sulted in some of the ugli­est code in the his­tory of man.

The parser and sug­gester

Cre­at­ing the parser and sug­gester, was rel­a­tive easy, but es­pe­cially the sug­gester has some se­ri­ous per­for­mance is­sues. The parser sim­ply re­turns a syn­tax tree from a string. The sug­gester re­turns a list of par­tial re­sults of pars­ing a string. There can, of cause, in some cases be in­fi­nite pos­si­bil­i­ties, there­fore is the re­turned list a lazy list. A lazy list, is a list of on de­mand com­pu­ta­tions. This means that all sug­ges­tions are not cre­ated in one go, but is de­layed un­til needed.

The sug­ges­tions are done by pars­ing the en­tire string. If the sug­gester reaches the end of the string mid rule, the rest of the ref­er­ences is filled with holes. Us­ing this tech­nique the sug­gester can in­fer "let ? in ?" from "l" alone. Given that the user did not like that, then the next sug­ges­tion is pro­duced by ig­nor­ing the first re­sult and con­tin­u­ing in the gram­mar. The de­sign of the gram­mar is di­rectly re­lated to the sug­ges­tions the user is get­ting. It's not al­ways the ideal so­lu­tion that is pro­duced by the sug­gester, but the ini­tial guess seams to be cor­rect most of the time.

The pretty printer

This was where I re­alised I had done it all wrong. Writ­ing the pretty printer was very er­ror prone and not es­pe­cially pretty. Pretty print­ing is done by trans­vers­ing the tree and then pro­duc­ing html. The only way to get the kind of node it was, was by match­ing with the syn­tax kind. The syn­tax kind is a tu­ple of the name of the rule and the num­ber of the ex­pres­sion that cre­ated the syn­tax tree. Match­ing on a string is in­her­ently hard, and the type checker does not give a lot of help. This ex­am­ple from math, shows the frag­ile match­ing needed to pretty print the tree.

case tree.kind of
  ("AddExp", 0) -> tree `take2` \a b ->  
    grp "add"
      [ a, operator "+", b ] 

  ("AddExp", 1) -> tree `take2` \a b -> 
    grp "minus"
      [ a, operator "-", b ] 

  ("MulExp", 0) -> tree `take2` \a b -> 
    grp "multiply"
      [ a, operator "⋅", b ] 

  ("MulExp", 1) -> tree `take2` \a b -> 
    grp "fraction"
      [ grp "numerator" [ a ]
      , grp "denominator" [ b ]
      ] 
   ...

Not only do we have to match on strings, which is slow, we are not guar­an­teed that the tree is well formed. This means that the take2 func­tion can fail and will pro­duce mal­formed pretty printed html (it will lit­er­ally print "mal­formed") if the tree is not well formed.

This prob­lem would of cause also ap­ply to other ar­eas, if the user would want to im­ple­ment type check­ing or small step se­man­tics in the ed­i­tor.

What were you think­ing?!

Well, I was dri­ven by a sim­ple idea; for com­put­ers to be able to help us they need to be able to un­der­stand us. The whole pro­ject has been lead­ing up to have an ed­i­tor that is able to write code like in the ex­am­ple be­low.

This is how pro­gram­ming should feel.

I think that this is a much more nat­ural way of writ­ing code, than the cur­rent tex­tural ed­i­tors. Get­ting to this point was not easy, and there is still a long way to a work­ing ed­i­tor. Be­sides the clear lim­i­ta­tion of not be­ing able to open and save files, un­do­ing, and copy-past­ing, there is a lot of things to be learned from this pro­to­type. I think that this pro­to­type il­lus­trates that mul­ti­lin­gual syn­tax ed­i­tors are pos­si­ble, and work­ing with syn­tax trees di­rectly is nat­ural when you get the hang of it. The ex­tended ben­e­fit of hav­ing syn­tac­tic rel­e­vant sug­ges­tions, gives a nice flow to the process of writ­ing code. Lastly I think that syn­tac­tic holes are para­mount for do­ing syn­tac­tic edit­ing.

The di­rec­tional move­ment in this pro­to­type is a lit­tle hard to fol­low, and es­pe­cially in dif­fer­ent dis­plays. A lot more work has to be done to make move­ment user friendly, but the list move­ment was a good start. Some­times spe­cial con­structs would be mis­con­ceived as list, which is con­fus­ing, and not all tools work on the lists in the same way. It might be that in the fu­ture a no­tion of lists has to be build into the syn­tax, which also di­rect move­ment. The most con­fus­ing part in the move­ment is when the depth of the tree changes vi­sual di­rec­tion. In the math ex­am­ple, the depth of the frac­ture is ver­ti­cal, while the depth of the ad­di­tion is hor­i­zon­tal. I have heard sug­ges­tions like a mini map dis­play­ing the de­bug tree, which could help you with the users bear­ing, or have vi­sual cues like color or un­der­lin­ing to show the par­ent and the near­est sib­lings. Even though that is it hard to move around in the tree I still think it is un­clear, if this is some­thing that you can learn when us­ing the ed­i­tor over time, and then you might even get more ef­fi­cient. Please try it out and see for you self! Any feed­back will is very ap­pre­ci­ated.

One of the biggest de­sign de­ci­sions was where whether to have sta­t­i­cally de­fined syn­tax trees or dy­nam­i­cally de­fined syn­tax trees. Where a sta­tic syn­tax tree de­f­i­n­i­tion would have the ben­e­fit of the type checker, both to do easy match­ing and to en­sure that all op­er­a­tions on the tree re­turns a valid tree. On the other hand does the dy­nam­i­cally de­fined syn­tax tree en­able op­er­a­tions that work across all lan­guages. When start­ing out with the pro­ject I felt that the generic syn­tax tree ap­proach was the most sus­tain­able, but af­ter I tried to do lan­guage spe­cific ac­tions like pretty print­ing I found that it was al­most im­pos­si­ble. It is clear to me that some kind of golden mid­dle way has to be found. I see a cou­ple so­lu­tions to solv­ing this. The most in­ter­nal way of do­ing it would be to spec­ify some core fea­tures that makes a syn­tax tree a syn­tax tree and then im­ple­ment­ing them for each lan­guage. This prob­a­bly re­quires some­thing like Haskell's type classes. An­other more dy­namic way would be to keep the dy­nam­i­cally de­fined tree and then de­velop a DSL for work­ing with trees. Both of these ap­proaches would be in­ter­est­ing.

Us­ing a lim­ited PEG as the gram­mar worked out well, but the tight cou­pling be­tween the gram­mar and the syn­tax tree were not ideal. If the syn­tax tree was sep­a­rated from the gram­mar, it could be able to ex­press more things. A PEG can only parse con­text free lan­guages. XML, and there­fore also HTML, is not a con­text free lan­guage. Choos­ing PEG as the only gram­mar would force our ed­i­tor to never be able to edit XML. Some gram­mars, from which syn­tax trees are made from, use de­com­po­si­tion, and refers to other rules to tran­si­tive in­herit their ex­pres­sions. To en­able the user to move around the tree with­out work­ing their way through in­fi­nite nodes that only point to other nodes, the tree has to be trimmed. I also found some places where the gram­mar was not suited for my sug­ges­tions al­go­rithm, and where the or­der of the rules had to be changed to get bet­ter re­sults. All of this sug­gest that the gram­mar shouldn't be the tem­plate of the syn­tax tree, but more be used as a parser, that can fill an ex­ist­ing tem­plate.

An in­ter­est­ing prob­lem with syn­tac­tic ed­i­tors is some­thing like com­ments. For a syn­tac­tic ed­i­tor to be able work with com­ments, it needs to be able to parse and un­der­stand the se­man­tic mean­ing of com­ments. Cur­rently there are con­ven­tions for where to put com­ments, but they are not en­forced by the parser. That needs to change.

One of the cool thing that Ash is able to do is the code sug­ges­tions, but the sug­ges­tion en­gine, did not work per­fect and the user could find them­selves forced to search through a long list of pos­si­ble end­ings to get to the piece of syn­tax they needed. In the early ver­sions of the pro­to­type that was the only way to ap­pend or in­sert. I changed that so that it used the holes ap­proach in­stead, but the sug­ges­tions are still in there. While the sug­ges­tions are valu­able in­for­ma­tion for be­gin­ners, who do not know the syn­tax of the lan­guage (Note: To il­lus­trate this point I have not writ­ten down the full syn­tax of the two lan­guages here — did you fig­ure out how to code in them?), it might not help ad­vanced users as much. For them it could be cool to do things like se­man­tic syn­tax trees. The trees would dis­play like the nor­mal syn­tax trees, but have build-in se­man­tics. The tree might only be syn­tac­tic cor­rect if it only ac­cess de­fined vari­ables. Sug­ges­tions in such a tree would be some­thing like a con­text aware auto com­pleter, which would be very handy even for ex­pe­ri­enced pro­gram­mers. By ac­ci­dent do the syn­tax holes also feels like the holes used in code syn­the­sis, where com­pli­cated heuris­tics would try to fill out holes in a pro­gram. This cor­re­la­tion leads me to be­lieve that syn­tac­tic edit­ing could be the gate­way drug to em­bed­ded code syn­the­sis.

Yeah, okay... What's your point, man?!

Code is not text. Code is lan­guage and should be treated that way. When the com­puter speaks the same lan­guage as you, you can be more pro­duc­tive. Syn­tac­tic edit­ing is not only pos­si­ble, it is prac­ti­cal. Cur­rently the tech­nol­ogy has a lot of rough cor­ners, but I be­lieve that syn­tac­tic edit­ing is the fu­ture, and I hope that I con­vinced you, or at least make you ques­tion the way we edit code now.