Opensim under the surface: May 2009

Adam Frisby's first monthly report

Article by:

Adam Frisby

With May coming to a close, I would like to start the first of a monthly feature - some explanations of what’s going on under the surface of OpenSim. But wait, you may ask; doesn’t Justin already provide excellent weekly reports on what’s been committed? Of course - where this differs is in the level of detail, particularly on infrastructure projects. Justin does a weekly feature which means he only has time and space to report on features in bullet form. Where I plan to differentiate with these posts is in the details - not only what has been committed, but also where it is going.

There has been a lot of progress this month - starting at revision 9358 there have been several major areas of focus.

Networking: LLUDP Improvements

We now support the Multiple Object Update packet format for Linden UDP clients (most notably Second Life & realXtend), this builds on work that Mikko of realXtend provided late in April that did the same for Avatar Updates. The work done by Melanie reduces the total number of packets that need to be tracked by packing multiple updates into each packet - resulting in a significantly lower workload for our packeting code. The Avatar packet aggregation with 100 Avatars in the region results in a massive 71% reduction in the number of packets transmitted, and a 29% drop in CPU requirements, according to tests done by Intel for ScienceSim. Object aggregation produces similar results (although hasn’t been measured precisely.)

Packet Pooling - the next major improvement is in ‘packet pooling’ - one of the drawbacks to managed languages is very little control over optimising garbage collection. OpenSim handles potentially thousands of packets each second, in our previous packet system for each of these packets, we created several classes representing the internal data structures. (eg, Byte[] Packet becomes MyPacketObject) - while the processing is fairly quick, creating these packets results in the Garbage Collector handling tens of thousands of tiny unused classes at each collection. On .NET this runs so-so, but on Mono it runs atrociously.

Packet Pooling solves this problem by recycling classes - rather than leaving a dead class to be garbage collected, we stuff new data into it and re-use the class instance. We previously did this work back in 2007 with the help of 3Di developers, however it was at the time proven unworkable due to problems with old data persisting into new data blocks. Melanie managed to solve this issue, and nearly two years later we have it re-enabled. Early pooling code did not show much promise past 40 avatars (although up to that point was fairly dramatically better) - however more profiling done by intel managed to locate the problem as being due to a finite sized pool - a patch fixed this problem and the overall improvement was a further 40% reduction in CPU requirements for 100 avatars (total reduction for this and aggregation is a whopping 57% reduction in CPU requirements for a busy sim).

Overall, the OpenSim networking code has been improved very significantly (especially with April’s texture resend fixes), and I would appreciate seeing more testing with large userbases - the Wright Plaza weekly meetings have been running better than they have for a long time, but only represent a peak of 35 concurrent users.

Some early bugs did present themselves with this networking code - most notably running into several MTU limits for international users (apparently most international trunk lines will simply drop rather than fragment big UDP packets), myself, Nebadon and Melanie managed to identify and fix this problem.

I introduced, then disabled support for NAT Loopback - this allows you to emulate a loop-back compatible router for those hosting from home and unable to get a public internet IP. This is currently commented out due to issues with compatibility in Grid Mode (however Standalone mode can be made to work effectively) - further improvements will be in the realm of adding UPnP support allowing you to host a region from home reliably. Finally I also introduced the capability to use IP and DNS based bans (including wildcards), Dr Scofield improved this with some stability checks.

---

Add this page to your favorite Social Bookmarking websites

According to OpenSim core developer Charles Krinke, there are 330 key functions in Second Life – and 300 of them have been fully implemented in OpenSim.

Of the remaining 30, half have to do with vehicle physics, he said.

These are the functions that allow virtual passengers to ride in virtual cars, for example, fly virtual planes, or race virtual horses.

Except for these functions, objects can be imported from Second Life to OpenSim, and work much the same way.

I’ve seen this for myself. Doors open and close, in OpenSim worlds, for example, just as they do in Second Life. My hair swings when I move my head.

I even have ducks swimming in my pond.

According to Krinke, animating objects happens through the use of scripts, and OpenSim supports the Second Life scripting language.

Except for those pesky vehicle physics scripts – and those are mostly done, he added.

“You can generally expect to get 90 percent of the functionality of a Second Life simulator,” he said.

The other major functions still missing from OpenSim are voice chats – as opposed to the typed kind of chats – and group functionality.

And I’ve seen all of these functions – vehicle physics, voice chat, and groups – being tested over the past few days.

In fact, according to Krinke’s fellow core developer Adam Frisby, the difference between OpenSim and Second Life is  no longer 5% but just 2%.

Read full article here

Article by:

Maria Korolov