jeudi 16 octobre 2014

Sirius 2.0 : Under the Hood

This summer was quite intense for the Sirius team. Sirius 1.0.0 was barely out that development of the 2.0 stream started while report from adopters which are not part of the Eclipse Release train started to appear.

That represents fixing more than 100 tickets since the end of june, a few for Sirius 1.0.1 which got published a few weeks ago (and is part of Luna SR1). From those tickets 82 are targetting Sirius 2.0 (over a current perimeter of 101 tickets), we are currently validating the changes and closing the gap. As a sidenote more than one third of the tickets implemented in this version are directly funded by end users (hint : this might be you).

Sirius 2.0 will be released just before EclipseCon Europe , most of the 21 enhancements in this scope are already implemented and are under validation:  now might be a good time for giving a look.
These enhancements are mostly focused on the diagram editor user experience, fixing long standing issues inherited from the default behavior of the GMF runtime and bringing nice features for intensive users of diagrams. But we'll tell more about that in a specific post, lets focus on the "not so visible changes" for now :

Performances and Scalability

Another strong focus of the 2.0 version was performances and scalability : finding bottlenecks and taking them down.

We worked on the basis of EcoreTools 2.0 with a "big enough" model to detect such bottlenecks and mostly focused our efforts on the CPU usage (for now but memory will also be important at some point too).  1.5Go heap was the target, as long as the scenario were doing fine within this bound we did not investigate.

What is a "big enough" model you will ask, it is composed of 500 000 model elements (here EClasses, EAttributes, EPackages ...). 20 000 representations are created on top of those, 2 of them representing hundreds of EClasses.

The test data is reverse engineered from the Sirius source code itself which make it quite "close" to a sensible Ecore model. Such a model makes it easy to identify bottlenecks: places where your code have a complexity relative to the number of model elements, or places where you are doing too much work on the UI thread.



This has proven to be a very efficient way to identify problems and to fix them. Having a consistent installation of EcoreTools + Sirius is quite easy, opening the test project become a breeze then and anybody in the team can reproduce the problem and measure for himself.

And here we are :


We fixed 16 tickets tagged "performances" for the 2.0 release, here is the list of improvements :
- Sirius is initializing itself quicker (notable on startups and first usage)
- Many calls from the UI threads which were scaling poorly when you have thousands of representations are now way faster
- Delete have seen a big boost and is now scaling based on what you specify and how many things have changed during the execution of your tool independently of the size of the model
- Diagram having lists with many elements are created and refreshed quicker (the improvement is noticeable has soon as you have lists with hundreds of elements)
- Select All used to have an irritable lag when you were working on a big model, it is now instantly completed.
- Tree Editors defined using Sirius are now more efficient in refreshing the SWT Components.

But more things have been identified often with partial patches which have not graduated yet to the stage of "reviewed and regression tested" but will in the next releases. Expect some improvements in the way "save" is handled when you have hundreds of resources. This is a constant effort but one which has great pay-off: any improvement in Sirius itself brings improvements in the dozens of tools which are relying on it.

Performances are always a tricky issue for a generic framework or technology, so many things will depend on how its used and in which context and the most innocent change can dramatically impact the properties of your tool. The #1 rule is : have test data which are representative of what you want to achieve, make sure it can be used and migrated easily from version to version.

Headless - aka reuse the plugins with no UI


Another area which have seen improvements is about using Sirius in "other contexts".

A first batch of changes has been merged so that some of the basic services offered by Sirius are now able to run without any UI. Things like loading a representation resource, creating a diagram,  refreshing it, modifying the model and saving can now be used as a server-side or continuous integration process for instance.

Here is a scenario which will work in headless :

 
I'm pretty excited about this release and I'm looking forward to see these improvements in the tools already adopting Sirius. Of course these are just a few more steps closer to the goal, Sirius 3.0 planned for June will bring even more !

jeudi 25 septembre 2014

What makes EclipseCon Europe a good conference ?

EclipseCon Europe 2014 is getting close and it will only get harder to book hotel rooms and flights. It's probably time to decide whether you come or not. Let me tell you what you can expect from such an event :


Getting in touch with the technology,  the people behind it and its users.
I would argue that this is the main point of such a conference. Those 3 days are filled with presentations either from the project commiters giving a glimpse of what you could expect, or from users of the technology giving experience reports about how it helped them and what you might want to keep an eye on. Furthermore, presenters are sticking around during the conference and the friendly spirit makes it very easy to start a discussion and learn more.
This content is something you won't find anywhere else.


A lot of Modeling content
If you are into modeling or using it at work, going to this conference should be a no brainer. Xtext, Sirius, EMF Compare and many more technologies are represented during those days.

Speakers are good
In the last five years we have seen an impressive boost of quality regarding the talks and the way they were given. Most of the talks are kept in a 35 minutes slot and with the Program Commitee we worked hard to make sure the sessions will have good content but also good speakers.

Tutorials,  Unconference
3 hours sessions are planned so that you can get your hands dirty using some of the Eclipse Technologies. IoT,Cloud , Java, OSGi, EMF or Xtext are covered for the 2014 edition of the conference. Not only that but rooms are booked for the "Unconference"  so that working groups can collaborate or projects can setup a "mini hackaton", that's a rare chance to gather people from different countries to work with.


Social Events 
The Circus, the concert, the receptions. Many occasions to enjoy yourself and to meet the Eclipse family.




If you haven't registered yet you might consider doing it now as the price will rise at the end of the month.  I'm looking forward to meet you there !

vendredi 20 juin 2014

EcoreTools 2.0 - The Luna Revival




With Eclipse Luna comes a complete re-implementation of EcoreTools, the diagram editor for Ecore. This is an important piece of news because EcoreTools is often the first step our adopters have to go through. Users are expecting to have a diagram editor to design a domain model. They are expecting this because they are used to this notation, they learnt it at school or university and even just this ubiquity makes it powerful in itself.




And that's why EcoreTools matter. And that's why EcoreTools matter even more for an organization like Obeo.

But then what happened ? Why did this project have been stale for years ?

How it all started


Let's come back to the creation of this project. The proposal has been submited in 2007, and in 2008 EcoreTools 0.8 was shipped with Ganymede.
At that time building such a graphical modeler was quite costly, even with the GMF runtime and Tools which were leveraged by the original team, going from a very basic diagram editor to a complete tool with a consistent user experience was a lot of work. And they tackled this work and had progress from version to version up to the end of 2009 when the company behind this effort got acquired and changed its focus. The project entered hibernation then.
Because it was a key component of the whole modeling eco-system, Ed Merks and I stepped up to take care of the project, leading it through the bare minimum so that it can be part of the release train and still exist. The plan at that time was to find other people interested in investing in it, unfortunately it did not happen and I can't blame them, as I said that was costly, an investment you could not do without clear gains to expect.

Then Thales and Obeo open-sourced Sirius within Eclipse : a complete tooling and framework designed to build rich modeling environment efficiently, also designed on our experience with the shortcomings of the GMF Tool project. Creating an Ecore modeling environment then became barely more work than maintaining its build. I already had a starting point as we built an Ecore modeler with Sirius before, but many things in this tool were not quite exactly as I wanted it. That lead me to rethink all the interactions to provide a tool which actually assist you in design a good Ecore model easily.

I started by inspecting the usages of "EcoreTools 1" or our own commercial modeler internally. At Obeo we are designing Ecore models all the time. We have customers which do need to design Ecore models often, we do consultancy projects in which the Ecore model is the backbone of all the tooling we are building, yet EcoreTools 1 was not used at all and our commercial modeler was almost exclusively used to produce diagrams for the documentation and deliverables. And when I digged deeper, I realized the tool was just barely helping compared to the tree editor provided by the EMF Project itself, and sometime was even "getting into the way".

I also wrote down scenarios of usage for the tool using personnas. Both these efforts have been enlightening : I could clearly identify usability problems and expectations regarding the tool. I came up with a few key points to focus on :

1 - making sure EcoreTools was providing more value to users knowing Ecore than the Ecore Tree Editor.
2 - making sure that EcoreTools will not make you "look bad" when using it with a customer

Ecore Support

EcoreTools 2 obviously allows you to design Classes, Datatypes, References and all the classical Ecore constructions, but Ecore is way more than that. There are different concerns involved in expressing your domain model: documenting, reviewing the referencing mechanism of elements, identifying business constraints, exploring the model... Specific tools have been implemented for these concerns and most of them are "opt-in" in a given diagram through the activation of a dedicated layer.

Documentation

Your Ecore model represents your domain. It's all about picking good names, but names are not nearly enough. Documenting your Ecore model is important and EcoreTools assists you with a dedicated layer named "Documentation" and a table editor to quickly go through all of your elements and document them.


Bi-directional references


With 2.0 the bidirectional references (also known as EOpposites) are displayed using the well known notation which comes from UML Class Diagram.


Constraints

Domain constraints can be listed directly from the diagram once you activate the "Constraints" layer. EMF will then use this information to generate all the required plumbing so that you just have to fill a java method to implement the actual check.




Generics


Ecore has been supporting Generics for quite a few years already, EcoreTools allows you now to express Parameter types and use them in references or operations.


Packages Dependency Analysis

Often domains have relationships to other domains, and this is reflected in Ecore models through direct referencing. But keeping track of those dependencies between domain packages proves to be hard in practice, you have to digg through every Class to see if it refers to something which is external. EcoreTools provides a diagram which is dedicated to see and analyze those dependencies.


And also a dedicated decorator for classes which are from another package.

Productivity

To be productive the tool needs to be at your fingertips. EcoreTools provides many shorctuts to make your life easier, especially regarding references and attributes. Typing "1" will make the reference or the attribute mandatory. Typing "*" will make it a "many". Just typing a name will only change the name, but typing ": someTypeName" will set the type.

You can't really discover those shortcuts on your own, so feel free to have a look on the documentation, but once you'll know them you'll be quicker in designing your Ecore using the diagram editor compared to the classical tree editor.

Many more things have been done to make sure the tool doesn't get into the way. All the references can be reconnected graphically, the modeler can be used in "full screen" with no other Eclipse views around while keeping a good usability, EcoreTools will take care of your GenModel too by reloading it when necessary.


Design and Feedback

From a graphical point of view I tried to keep the original visuals, only make them slightly more appealing. Colors have been aligned to the Eclipse Standard palette. Classifiers now have rounded borders, icons and text style are used to convey the difference between Classes, Abstract Classes and Interfaces.
Boldness is used for anything which is mandatory, blue for anything which is derived.
These are the kind of things you can think of if you can quickly turn-around and try in your graphical modeler, and Eclipse Sirius enables that.



A tool is helpful when it is giving you the right information at the right time. This is all about feedback.
EcoreTools 2 is highlighting in red any construction which is not valid. You want to know that as soon as possible.

But feedback is not always immediate, only if you enable the "documentation" layer to annotate your model, then the red borders will be used so that you can quickly identify elements which have no documentation.



How much work was that ?

Of course it is slightly biased because I know Sirius really well. I was actually part of the team building it for years now so anything I want to do in EcoreTools I can quickly see how I'm going to do it leveraging Sirius.

Nevertheless here is a typical change introducing the constraints support in EcoreTools, this is mostly about expressing, in the Viewpoint Specification Model of Sirius, what you want to achieve.

















No code generation is involved, my plugin defining the modeler is just a standard Eclipse plugin, I can use JUnit, SWTbot, Tycho, nothing fancy here is imposed to me by Sirius. Here are a few stats :

The diagram and table editors of EcoreTools 2 are representing less than 2700 lines of code.

Now what ?

This is the 2.0 version, freshly built for you. I think the net gain compared to the 1.x stream is already huge but of course the paint is slightly fresh, you might find issues and if you do please report a bug. If you like it, please tell me through twitter, google plus or the Eclipse Forums, this will be appreciated.

You can install EcoreTools using the Eclipse Marketplace, you will also find it in the Modeling Package which is shipped with Luna.


mardi 19 novembre 2013

Eclipse @Devoxx

I'm back from a full week at Devoxx in Antwerpen- Belgium. I was there to present the Sirius project and
Eclipse Modeling at the Eclipse Foundation booth. (by the way, thanks again to the foundation staff !)
It was quite fun to be there with Jelena Alter, Marcel Bruch, Julien Vermillard and Gael Blondelle, we had a good mix of things : Marcel for the Java developpers with Code Recommender, Julien for M2M stuff (which was very hot at Devoxx this year) Jelena and Gael for the Eclipse Foundation itself and me with fancy graphical modelers.





Here are a few random notes :

The conference organization and setup is quite amazing. Wifi worked very well, the venue is a theather complex which means you always get to sit very confortably and the screens are just huge. There was some hacking spaces with peoples hanging there all the time, voting was easy thanks to a bunch of arduino modules installed in each room. Interactions were encouraged with tweets being displayed on every screen in between the sessions, "free to use" whiteboards were positionned in the hallway leading to some wild polls. I liked these small things which are triggering more involvement from the audience.

The content was good or very good in general but could not attend many sessions as I had many things going on at the booth too, but overall I was impressed. It's not perfect though, I had my share of "sexy looking talks" which did end up being very badly presented.

In the end I have seen only a few Java talks, many sessions were related to web dev, cloud, or the now famous "Internet of Things".




I was there as an Eclipse guy and believe me, the Eclipse hoodie is like a secret weapon to get to talk to pretty much any programming rock star. On the other hand there is this trend going on in the Java community about Eclipse being really bad compared to the competition which made me a bit reluctant first, will people attack me there ?

After speaking with many Java developpers during the conference here is my report : most of them are loving Eclipse, as an IDE.
Juno was a lot of pain though and they did not understood why it got released with such poor performances and behavior. They like Eclipse Kepler way better but it's still not exactly on par with what they used to expect, and they expect a lot from Luna, in particular :

  • Performances and responsiveness
  • Quality and reliability, no more ui glitches in the workbench.
  • Java 8 support.
  • Maven integration (Many people have given up on this one starting from Juno...)

In the end the differences between the major IDEs are quite small and we have room for improvements in several points, but users have pain with IntelliJ too.

What about innovative features ? Actually Code Recommender in itself impressed many peoples, and the others could quite easily be impressed with features which have been around for years through specific plugins they did not knew about.

In a way that was kind of a relief for me to see that our work was not completely rejected by the community and that people could say things like: "Eclipse is the best for this, or that" (for instance having support for multiple languages in the same IDE).  Also, several sessions presenting very cool stuff that could not have existed without Eclipse.

But on the other hand it is very hype to reject and bash Eclipse when you want to brag as a speaker.

Of course it was also a great occasion try a few ideas discussed here and there lately. I realized that :

  • "Code Recommender" in itself makes people wanting to use Eclipse again for Java.
  • most of them don't understand what they are downloading from Eclipse.org and the whole idea behind the release train. They could not really figure out which "thing" they should download between "Classic", "Java" or "JEE".
  • most did tell me they downloaded "Eclipse" (which means, for instance, the Java package) and were surprised that it was not having the X or Y feature - which actually is part of the release train.  They just have no clue how to discover that.
  • nobody wanted the "uber package" : that would make an un-usable IDE and they don't want feature they don't use to interact with others (and the bloat)
  • the "let's display fake wizards which are provisionning the IDE on demand" was midly received. Most people I talked too want to be sure they won't need to install something more once they prepared their setup. 
  • a "configurator wizard" opening up the first time so that the developpers gets to pick what kind of langage support he wants, which SCM integrations, which bug tracker and so on, kinda excited the crowd. They see that as the perfect way to provision just the Eclipse they want withouth needing to figure out what all these weird feature names are meaning and knowing this is a selection of high quality plugins, all of them being open-source.
  • only a few would be willing to pay for Eclipse, even if it is a special version, and even if it's a very low price. They are used to Eclipse being free and are more willing to click donate than to buy the binary. By the way, they feel like when they click on this donate bouton, they are already helping the development of Eclipse as an IDE.
  • many features of Eclipse, even if they can be installed through the release train and can be usefull for a developper, are not known at all. (the configurator wizard could help here too). This is, again, probably the smallest effort on our side to have the highest impact.
  • several were using Eclipse as a platform, to build applications or tools, and they could not find the same level of flexibility and extensibility in any other platform out there.

This is it for "Eclipse as an IDE". 

Regarding Sirius, I had very good feedback. People were either curious about it or left with the idea of trying it in their own context, which was more than I could expect from such a conference !



mercredi 2 octobre 2013

Eclipse Modeling Package - The Road to Luna

Since the last public survey, my primary focus for the modeling package was :

  • to include Ecore related technologies, or companion technologies with a low UI profile
  • to only include components which are used largely enough to consider them as stable (and not in incubation)
  • to ease the discovery and installation of the other Eclipse Modeling technologies
  • to include what is necessary (Git,Java and plugin development, sdks) to develop your own specific tooling using Eclipse Modeling.
As the package was fairly large at that time one of my primary course of action was to try to keep it slim. Not that I think the size matters that much in the end, but the more we add in the package, the less things works as expected and as testing and validating the package is a one man effort, I had to make choices based on that too.

Several people chimed in lately with other components to include - and as we are starting the Luna cycle I think it is a good time to have another survey and learn about what you expect as an end user.

Please, take a few minutes to answer the following survey, tweet about it, share the link on google plus or whatever so that I can get data to push things forward.