NGVM Think Tank Meeting Notes
Dec 6 & 7 2005
A small group met for one and a half days in San Francisco to discuss a next generation JVM project and brainstorm ideas about Java and JVM design and implementation. The following is a short summary of that meeting and its outcomes. Unedited meeting notes are attached.
Attendees
Brian Bershad, U Washington
Steve Blackburn, Intel
Hans Boehm, HP
Michal Cierniak, Google
Cliff Click, Azul
Daniel Frampton, Australian National University
David Gregg, Trinity College, Dublin
David Grove, IBM (via email)
Xiaofeng Li, Intel
Bernd Mathiske, Sun (first day only)
Glenn Skinner, Sun
Background
This meeting was called with the goal of drawing together experts who are interested in the development of an open next generation JVM. The particular goals of the meeting included brainstorming the context for developing such a VM (the challenges, issues, etc), and discussing the form such a VM and VM development effort would take.
Brainstorming
The first half of the meeting consisted of a series of brainstorming sessions, with subjects ranging from the high level “Where is the biggest future potential for Java?” to the more detailed “What should be the key points of policy flexibility within a JVM?”.
We brainstormed this theme at length, primed with 5 questions. For each question the group came up with a large number of ideas and then attempted to group the ideas into themes, generating a “mind map” for each. The questions and responses are briefly summarized here, with links to the mind maps, and the number of ideas associated with each theme indicated. The mind maps serve as unedited (and thus not necessarily intelligible) records of our extensive brainstorming.
1. Where will (or should) Java be in 10 years?
Think about Java, where it is used, how it is used. Here we want to think about where Java might be or could be in 10 years. Of course we may well want to think beyond Java. Here we want to be very blue-sky, critique the language, crystal ball about applications etc etc.
· What are the most important new and future application domains for Java? (services 6, systems programming 3, real time 2, multimedia 2, dynamic languages 2, small devices 2)
· What are the most significant limitations of Java? (language specification 11, performance 6, JVM issues 4, reliability 4, class libraries 3, systems programming 3)
· What are the biggest future challenges for Java? (evolution 8 (too slow 5, too fast 2), performance 6, competition 3, new programming models 2, fragmentation 2)
· Where is the biggest future potential for Java? (dominance of Java platform 8, software engineering 5, new application domains 4, runtime optimizations 3, everything is managed 3)
· What are the things you would most like to add or remove to/from Java? (software engineering 8 (modular design 4), improved class file formats 4, concurrency 3, performance 3, isolation 3, systems programming support 2)
Persistent themes included the language specification, performance, systems programming, and Java’s role in supporting good software engineering. The lack of good support for native code interoperability was mentioned a number of times (both with respect to data types and calls). The two other major themes to arise where: a) the dominance of the Java platform (and the extent to which it would become universal), and b) the challenge of evolving the Java platform.
2. What should a next-generation VM look like?
Many of the second generation VMs have evolved from code bases 5 to 10 years old. Here we want to think about what a VM might look like, given the chance for a completely fresh start. We want to think forward about new requirements for new VMs, and also how a fresh start could better serve current needs.
We changed our brainstorming format, reducing the number of questions and ideas, and instead spent more time on identifying themes. After identifying themes, we noted those that were most contentious (of course there were many important themes which were not contentious).
· What are the biggest limitations of current VMs? (implementation issues 8 (lack of modularity 4, relationship to OS 1), VM specification 6 (only runs Java 2, no isolates 1), performance 3
· What are the most important lessons learned from current VMs? (performance 9 (specific optimizations 6, must use inline caches 1), flexibility 4, portability 3, VMs are effectively OS kernels 1, Writing VMs is hard: you need experts 1, do not add features too quickly 1.
· What should be the highest priorities if building a VM from scratch? (construction 11 (robustness 4, flexibility 2, java-in-java 2 (java in java bootstrap 1), core APIs designed by the few 1), feature set 7 (predictability 1, light weight 1, isolation from outset 1, resource management 1, correctness of application 1), survival 6 (building a community 3, make evolution easy 1), universality 1.
A number of issues stood out in this session, including lack of modularity in the VM implementation, the importance of performance to VM implementation, the relationship between the OS and the VM, and difficulty of writing a VM. The first two of these were widely agreed upon. The second two generated a lot of heat. Most notably there was a long running discussion on who should design a next generation VM, the difficulty of the task, the need for experts, and the compatibility of this situation with conventional open source development models. Some consensus seemed to emerge that designing and implementing a product quality JVM required a great deal of expertise, more so than typical product-quality implementation projects. The question remained open as to how this should play out in an open source development context, and particularly, in the context of Apache. This issue was underlined by Dave Grove in his email response: “building a high quality VM requires people with highly specialized knowledge & serious system programming ability. There aren't that many people with both and many of them work on commercial VMs. Also need continuity (or extremely disciplined documentation of design/architecture) since things are so inter-related.”
3. Innovation
If Java and more specifically, the VM’s we’re thinking about, are to be significant and successful in 2015, there will need to be a lot of innovation between now and then. Here we want to think about innovation in Java, both with the language and with the technology such as VMs. What needs to be done to maximize innovation?
In this session we reduced the number of ideas each person contributed and reduced the time available accordingly, so the maps are less dense.
· What have been the most important sources of innovation for Java and VM technology to date? (industry 8 (application or market 2, rich Java ecosystem 2), experience 4, research community 4.
· What has held back innovation in VM technology (generally, and in your specific experience)? (complexity of Java spec 5 (lack of good open source VM 1, lack of class libraries 1), non-technical 4 (licensing issues 2), complexity of existing implementations 3, success of Java 3, platform constraints 2).
There was a lot of unanimity here. It was widely held that industry had been the major source of innovation and complexity of both the JVM Spec and existing implementations had been major roadblocks to innovation. Dave Grove added: “many optimizations that work well on small benchmarks (SPECjvm98, SPECjbb, etc) don't directly scale out to large "real" middleware workloads. Catch 22 of doing work in easy benchmarks, and then hoping to get lucky or doing work with "real" work loads, but going much slower due to inherent pain/ugliness of the work loads.”, “Academics need infrastructure that is close to production quality, but more malleable (and smaller)”, and “Do better job of getting workloads between jbb2005 and jAppServer for researchers to use.”
4. Design
Performance is obviously key, and yet the ground rules have changed dramatically over the past ten years and will continue to change. Here we want to think about the highest priorities for a new VM design, with regards to performance considerations.
Flexibility is often in tension with performance, but flexibility can facilitate innovation and greatly improve productivity (and thus, indirectly, performance). Arguably, most second generation VMs lack flexibility. Here we are talking both about deployment (J2ME, J2SE etc), and innovation (varying internal policies, design choices & mechanisms).
· What hardware features would be most valuable to VMs? (control flow 5, memory management 4 (tagged pointers 1), concurrency 4 (full/empty bit 1), ISA changes 3 (direct bytecode execution 1), observability 3 (PMU 2), isolation 2 (lightweight sandboxes 1, fine grained protection 1))
· What should be the key points of policy flexibility within a VM? (VM internals 13 (execution 4, code translation 1), modular VM spec 3)
· What are the best techniques for reducing the tension between performance and flexibility? (build and execution strategy (partial evaluation 7 (easy to extend code gen 2, java-in-java 2, high level specification language 2), good design 6 (design for ideal case 1))
· What are the keys to ensuring robustness? (design 10 (care with dynamic behavior 3), testing 8, recovery 4 (transactional memory 1), application robustness 3, debugging 2,
Control flow, memory management, and concurrency dominated the discussion of potential hardware features. The value of a full/empty bit (see Burton Smith, Terra, HEP & Alewife) or similar fine grained per-word synchronization facility was hotly discussed for a while. The value of fine grained hardware protection was also hotly debated, with the majority feeling it was not of great use to a JVM. Choice of execution strategy was the largest theme in terms of design flexibility. Careful design and partial evaluation such as dynamic specialization and the use of java-in-java were the major themes for reducing the tension between flexibility and performance (as Dave Grove put it: “Must assume "super-human" inlining.”). The group identified two prominent themes with regards to robustness: the need for a rigorous, extensive testing strategy, and the importance of limiting and controlling non-determinism. The group viewed non-determinism, particularly with regards to dynamic optimization, as the greatest foil to robustness in modern VMs.
In addition Dave Grove has contributed the following via email:
“VMs are composed of what seem like fairly independent components (core, JIT, GC, etc), but the interactions between them all are subtle and numerous. Especially if you care about performance, none of these components can really be designed in complete isolation from the others. But, as much as possible you do want to isolate the components to enable most developers to not have to know intimate details of the whole VM before they can do work in one sub-system. Hard to get "right" and constant source of pain. Trade offs between performance and flexibility abound. Object model cross cuts the entire VM and subtle aspects of that tend to leak into all sorts of dark corners.”,
With regards to scalability: “Have to build this in from the beginning. There's nothing more painful than removing the "one big lock" that serializes access to some core VM data structure. Been through this several times in Jikes RVM and on other projects. Scalability can't be an afterthought. Locks need to be as fine grained as possible (but no finer!), and you have to think hard about how subsystems are going to interact. Also have to think about weak memory model issues (not all hardware is as forgiving as x86)”
And: “JVM code base should be thought of as generating a family of VMs, not just one. Adds signifcant complexity to code base & development process, but I don't know how else to get a "single" code base to go from embedded devices to large scale SMP servers. Separate JVM from OS/architecture details as much as possible (but, may cost you in performance since you end up with LCD OS interface and some fancy tricks won't be there), but not too much (allow for some OS specific knowledge where it makes a big difference).”
Discussion
After the lengthy brainstorming sessions, we moved to a discussion mode. We kicked this off with a round-table session where each attendee had four minutes to answer the question “What would your priorities be if you were tasked with building a new JVM from scratch (given appropriate resources)?” These discussions provoked four questions which became the subject of the remainder of the meeting.
1. What will NGVM be “the most of”? (in other words, “How will NGVM differentiate itself?”)
Steve’s response to this question was that it should differentiate itself from other VMs by being the most “innovation friendly” JVM—a VM which facilitated innovation (and thus most likely was innovative itself). This immediately raised concerns about the objective of being product quality (performance, completeness, reliability). Shouldn’t that be our first priority? The answer was clear that being product quality was undoubtedly our first priority. We hope to match or better the performance, reliability and completeness of commercial JVMs. However, innovation will be the point where we most strongly differentiate ourselves; what makes NGVM different. We then discussed what the objective of innovation entailed, and in a nutshell, it means componentization, and a big departure from the monolithic structure of most JVMs today. We discussed how we could measure the success of the project with respect to this goal, and agreed that it would be hard. Comparisons with Jikes RVM pointed to publications as an indicator of how well NGVM succeeded as a platform for innovation, and capacity to remain competitive was another (Jikes RVM has done well on the former, but is slipping on the latter).
2. Why should Harmony care about NGVM?
We broke this question down into two sub-questions. Why should developers care, and why should end-users care. The answer to the former is clear: developers want to innovate. This is true of industrial developers, academic developers and the larger open source community. The most optimistic aspirations of the Harmony project depend on a highly innovative base, which is what NGVM is aiming to provide. We digressed to talk about the challenge of designing something as ambitious as NGVM on an Apache mailing list, and the issues of maintaining vision and a coherent objective within that setting.
3. Where are the developers and who needs to innovate?
We identified two major groups of developers targeted by NGVM, those building products and those doing research. We decided that it was hard to pull out the open source community as a developer pool because the question of whether the software license was open or closed was less important than the developers’ goal (eg build a product-quality VM). We again discussed the technical challenge associated with building a VM. The point was made that a leading VM incorporated an enormous amount of new technology (the Jikes RVM publications page attests to this), and so was less about building functionality than about researching and developing entirely new technologies. This distinctive characteristic of VM development appears to create some challenges with regards to the conventional Apache development model.
4. What should the management model be?
We agreed at the beginning that compatibility with open source and industrial development needs was crucial, and that therefore the Apache license was obviously a place to start. We discussed legal issues, and agreed that the rules being used by the Harmony project were probably a good framework to start with. We spent a lot of time discussing the management model. The challenge was to create a development model which a) ensured that a coherent vision for NGVM was maintained and refined, b) that code development remained consistent with the NGVM vision and c) that there existed a clear mechanism for direct input from experts unable to be directly involved in code development. The subtext of this discussion was that NGVM needed to a) draw on established experts, and b) have a clear, coherent development plan in order to achieve its very ambitious goals. We spoke at some length about how best to reconcile this top-down management model with the typical bottom-up (committer-driven) open source model. We discussed the Jikes RVM model which is depends on a small steering committee and a community of committers.
The consensus was that the Apache license is attractive but that it was unclear how the management model would work. As a group we agreed that we needed to better understand the Harmony project and the Apache management model. Steve undertook to engage Geir Magnusson in the discussion, and to investigate getting the NGVM code base into the open as soon as possible.
We also talked about dependencies on class libraries. It seems clear that NGVM should componentize the class libraries just as it does the JIT and GC, allowing various class libraries to be plugged into the VM.