Performance Comparison: Openspace Sims versus Regular Sims

There has been endless conjecture on the Second Life forums about the true capacity of openspace sims versus their "full sim" counterparts. We know that up to four class 5 openspace sims run on a single core in a four-processor Second Life server, whereas regular sims each have their own dedicated core/processor. This generally means that up to 16 openspace sims will run on a single physical server, versus 4 regular sims. It is assumed (but I don't know whether anyone has confirmed) that regular sims and openspaces run on different servers; that you won't see a mix of openspaces and full sims on the same box. We also know that openspace sims are allocated less physical memory per instance than a full sim. But in terms of actual, tangible data, we don't have much else.

Residents with a lot of experience on both kinds of sims seem to agree that script performance is generally worse on openspaces, and that sometimes openspaces can experience "mystery lag" in the absence of any obvious cause. (It is thought that this is due to the increased number of "server neighbors" that openspaces have.)

I have conducted a series of tests on two class 5 sims, one openspace, and one regular sim. While the tests do not address the differences between how the presence of avatars or intensive physics operations affect sims, they do provide some insight into the script capabilities of the two kinds of servers, and hint at the effects of server neighbors.

Testing Methods

• All benchmarks were performed by running the Standard test suite on Mysti's Benchmark Sim Tester v1.1 by Mystical Cookie. The benchmark tester performs a number of specific script-intensive tasks and calculates an overall score based on how quickly the sim performs each task. In all cases, lower times and scores are better. Note that all tests were done under Second Life server version 1.22.4.90499, over the weekend of July 19-20 2008.
  
• To eliminate variance caused by avatars visiting the sim, access to the sim was limited only to my personal group, and I would teleport away after beginning tests.
• To keep any attachments from interfering with the tests, I visited the islands in "Test Female" form, which includes no HUDs or other attachments.
• Both islands are standalone sims with no other adjacent regions, meaning they weren't sharing any data with other sims, and were not subject to the performance hit of "child agents" from nearby sims.
• To account for any variation due to "server neighbors" (other sims sharing the same physical server as a sim undergoing testing), I restarted each sim after performing each test, and I tested each case (empty versus object-populated) three times. Any visible spike in a given test metric could likely be attributed to laggy server neighbors during that particular round of tests.
• For the object-populated tests, I placed an identical selection of items in each sim. I chose items that I knew from experience as a landlord use a fair amount of script time. For example, temp rezzers, disco balls, and texture-cycling prims. (I did not want to shame any designers in this post, so I am not publishing my list of specific items. I can provide the list upon request.)
  
• Before choosing which items I'd use for the object-populated tests I visited a random sample of 10 of my own openspace sims to see how many scripts the average occupant was running, and what the average "Script Perf" metric was. The script performance metric indicates how many script instructions per second are being executed on the sim, and is the most important number to watch when trying to determine the general script activity on a given island. I kept placing scripted items on my test openspace sim until I had reached the average Script Perf number. I then duplicated this build on the regular sim.
• While it likely doesn't have any bearing on the test results, both sims had identical parcel and estate settings, as well as the same terrain (default island design #2). I also set my client to Low graphics settings so the sim wasn't burdened sending a lot of object data to me like happens when view distance is turned way up.

Summary of Results
You can examine my raw data here. The results I present below are the average scores for each battery of tests. Note that the "Empty" score for a given test is for when the tests were run with no objects other than the tester itself in the sim, and the "With Objects" score is for tests run with objects in placed (the exact same set of objects in each sim).

Regular sim (lower scores are better)

• Mysti Score: Empty - 1,600, With Objects - 1,661
• Rez Check: Empty - 0:00, With Objects - 0:00
• For Loops: Empty - 3:24, With Objects - 3:30
  
• Lists: Empty - 3:56, With Objects - 4:03
• Link Msgs A: Empty - 3:43, With Objects - 3:49
• Link Msgs B: Empty - 3:42, With Objects - 3:49
• Encryption: Empty - 6:12, With Objects - 6:33
• Object Chat: Empty - 5:36, With Objects - 5:50
• Land Scan: Empty - 0:04, With Objects - 0:04
• Total Time: Empty - 0:26:40, With Objects - 0:27:41

Openspace sim (lower scores are better)

• Mysti Score: Empty - 1,495, With Objects - 4,397
• Rez Check: Empty - 0:00, With Objects - 0:00
• For Loops: Empty - 3:00, With Objects - 8:40
  
• Lists: Empty - 3:35, With Objects - 10:42
• Link Msgs A: Empty - 3:44, With Objects - 10:37
• Link Msgs B: Empty - 3:44, With Objects - 12:09
• Encryption: Empty - 5:10, With Objects - 14:49
• Object Chat: Empty - 5:36, With Objects - 16:09
• Land Scan: Empty - 0:03, With Objects - 0:08
• Total Time: Empty - 0:24:54, With Objects - 1:13:16

Observations

• After I had set up the duplicate builds on both islands and confirmed that each island had the same number of prims and scripts running, I immediately noticed that the openspace sim had a "Script Perf" rating three times higher than the full sim (14249 to 4590). Since full sims are supposedly more powerful than openspaces, I had expected the regular sim to be the busier one, since it would have more processing power to devote to scripts. But I wonder if this means that openspaces are tweaked to devote more of their resources for script processing than full sims are, since script performance is the most obvios limitation of openspaces? The benchmarking results would seem to corroborate this theory, as the openspace sim actually fared better than the regular sim in the "empty sim" tests.
  
• After every sim restart, I recorded the sim ID and IP address as reported by Help > About Second Life. By doing this I confirmed that after every restart both the sim ID and IP address did change. (There had recently been some conjecture that user-initiated restarts did not move the sim to a different server, but these tests prove that sims do in fact move when restarted by residents.) Note that I did not restart either sim between tests 3 and 4, as indicated by the raw data. This way, when you look at the last empty test results (test 3) and compare them to the first "with objects" results (test 4), you know that any difference in score is due to the object/script placement, not a change in server neighbors.
  
• In one case, my regular sim actually jumped to a different co-location facility! I confirmed this by doing a tracert on the old and new IP addresses. This was very surprising and isn't necessarily good news for estate owners with lots of adjacent sims, as border-crossings between sims in different co-location facilities can be painful.
• The regular sim tests did not indicate any major issues related to external factors (server neighbors?), as there is rarely as much as 6 seconds of variation on the score for any individual test. The difference across test passes was more pronounced with the openspace sim (the shortest test pass was almost 17 minutes shorter than the longest one), which may indeed confirm the theory that openspace sims are more likely to suffer the effects of laggy server neighbors.
  
• The additional script load on the full sim during the "with objects" tests had only a trivial impact on the sim. The average total test pass for the full sim was only a minute longer when the sim was populated with objects. This is in stark contrast to the nearly 300% performance hit the openspace sim took under the exact same script load.
• Even though the openspace sim scored significantly more poorly on the "with objects" tests than the regular sim, its dilation was generally 0.99-1.0, sim FPS was routinely 44-45, and the sim was easy to navigate with my attachment-less Test Female avatar. This is good because it means that while script performance in a given openspace might be sub-par, the sim itself can still be perfectly usable, so long as script responsiveness is not a major factor in one's enjoyment.
  

Conclusions
Based on the final results, I believe we can safely conclude the following:

• Since an empty openspace sim actually performs a little better than an empty regular sim on the script benchmarks, it appears that openspaces are tweaked to allocate more processing power to scripts to make up for their inadequacies.
• When running under a normal script load (based on a sample of currently-in-use openspace sims), script performance of a class 5 openspace sim is nearly 1/3 the performance of a regular sim. (Hey, it could be worse!)
• Since the "with objects" tests barely affected the full-sized sim's results, while they profoundly affected the openspace sim's final numbers, it's clear that full-sized sims have a lot more headroom for script processing; these tests didn't even touch a regular sim's threshold.
  
• Unless the large discrepency with test 5 on the openspace sim is attributable to some other factor (running on newer/faster hardware than the other tests?) the influence of server neighbors (other sims residing on the same physical server), seems able to have a non-trivial effect on openspace sims, while the interference appears to be less marked on full sims.
• Performing a user-initiated restart on a sim (openspace or full) does land the sim on another physical server most (if not all) of the time, and may even land you in another co-location facility.
  
• Even though the openspace sim scored poorly compared to the regular sim once scripted objects were added to the mix, the fact that sim FPS and time dilation remained admirably high shows that an openspace sim can still be perfectly usable and otherwise "healthy" even as script performance begins to suffer.
  

Work Remaining
These tests primarily focused on raw script performance and the influence of server neighbors, and did not cover things like intensive physics operations or anything related to avatars. I am not personally equipped (or motivated) to investigate these avenues, but I hope someone else out there is. *winks*