Prevayler Serialization Optimization

Question

I'm working with Prevayler, doing some very simple stress testing.

On my program, I can get 10,000 reads in 500ms, but writing or changing one object takes 23ms. (I don't have detailed specs on this computer. Nothing impressive. Dual core 3.0ghz, 7200rpm HD, and likely 1333mhz ddr3 memory.)

Serialization seems to be the bottleneck.

What can be done to increase write/change performance?

JProfiler I ran the program in JProfiler (10,000 User Objects being written to disk.), it ran for 250 seconds, of that time 99.6% was used in running org.implementation.PrevaylerImpl.execute

UPDATE I switched over to XStreamSerializer, which increased write speed to 21ms per write, and decreased storage from

3555KB journal/10,000 Objects -> 2421KB

350KB snapshot/10,000 Objects -> 313KB

I would like to see 1000+ writes/second.

---- Main Method ----

Transactor trans = new Transactor();

    for(int i =0;i<10000;i++)
    {
       trans.createUser("Username", "PasswordHash");
    }  


---- Transactor Class ----

public void createUser(String Username, String PasswordHash) {    

    String id = Helper.randomID();

    //Avoids id duplicates, id is 7 digit base58. (Like Bitcoin, but shorter.)
    //ID generation takes .0078ms.
    while(getUser(id)!=null)
    {
        id=Helper.randomID();
    }

    prevayler.execute(new CreateUser(Username, PasswordHash,id));
}


---- CreateUser Class ----

public class CreateUser implements Transaction, Serializable{

    String Username;
    String PasswordHash;
    String id;

    public  CreateUser(String Username, String PasswordHash, String id)
    {
        this.Username = Username;
        this.PasswordHash = PasswordHash;
        this.id = id;
    }

    public void executeOn(Object core,Date date) {
          ((Core)core).store(new User(Username, PasswordHash), id);
    }
}


---- Core Class (Store method) ----

private final Map<String,Object> dataMap = new HashMap<String,Object>();

public void store(Object object, String id) {

    dataMap.put(id, object);

    //Adds User id to a seperate ArrayList<String>
    if(object instanceof User )
    {
      userList.add(id);
    }
}

Could you please format the code? I am having hard time reading your code. Where is the class? and is `CreateUser` an inner class? What class are you serializing? — Multithreader, Aug 20 '13 at 13:38
CreateUser was originally an inner class just for my own convenience, and it is the one that is being serialized. Prevayler uses "executeOn" method to serialize the transaction to disk. I made some changes, is it easier to read now? — user2699942, Aug 20 '13 at 13:47
I would probably writ a small test case and use a profiler to check what the problem really is. — Thomas Mueller, Aug 20 '13 at 13:53
Are you transaction logs empty by the time you start this test? — Edwin Dalorzo, Aug 20 '13 at 13:56
The journal? It's empty when I start, then has 3555kb at the end. — user2699942, Aug 20 '13 at 14:40

score 3 · Answer 1 · answered Jan 27 '14 at 12:53

23ms is about right for one spinning disk. With Disc Sync, I guess a file is opened, the transaction logged, flushed to disk and file closed. If the disk head has to move for that (updating the directory information for instance), you look at those kinds of times. Random access to a sector of the disk is about 20ms, perhaps half of that for highest end drives.

It is often better to just flush to disk 10 times a second, then you can use that disk sectors are large enough to hold multiple transactions and that disks are very good at writing consecutive data, i.e. logs.

user2699942 · Answer 2 · 2013-08-21T12:18:57.227

I turned off Disk Sync (Which is on by default.), now I'm getting 2500 writes/sec on this computer. With an SSD I was getting 10,000 writes/sec. (Reads are much faster too.) That works for me, but I'm not sure why Disk Sync slows it down that much, maybe someone can speak to that?

public Transactor() {

    if (prevayler == null) {
        try {

            PrevaylerFactory<Core> factory = new PrevaylerFactory<Core>();
            factory.configurePrevalentSystem(new Core());

            //This
            factory.configureJournalDiskSync(false);

            factory.configureJournalSerializer("Journal",new XStreamSerializer());
            prevayler = factory.create();
...

Prevayler Serialization Optimization

2 Answers2