Michael L Brereton - 02 February 2008, http://www.ewesoft.com/
<<
Previous: Database Basics
Setting Up A Database For Synchronization
The eve.database.Synchronizer Class
Synchronizing Across a RemoteConnection (EveSync)
In the previous chapter, use of the Ewe Database API was described and the technique of using a Java object as a means of easily reading and writing records to a Database. In this chapter we will discuss the Database API in more detail.
The DatabaseEntry object is the actual object used to read and write the data of each record. It is responsible for encoding and decoding the fields into bytes for storage and retrieval. You write field values into the DatabaseEntry using one of the setField() methods (e.g. setField(int fieldID, String value)) and retrieve values using one of the getField() methods. deleteFieldValue() will delete a field from the entry. After setting field values you call save() to write the DatabaseEntry to the database.
There are a number of other methods that you can use and these include:
byte [] encode() is used to convert an entry into a byte array suitable for transmission.
void decode (byte [] bytes) is used to convert a byte array back into a DatabaseEntry.
void duplicateFrom(DatabaseEntry other) is used to duplicate another DatabaseEntry completely.
There are a number of special pre-defined fields that you can add to your database using addSpecialField() most of which will have their data automatically set as appropriate. The ones that you may most likely need to use are:
OID_FIELD – This is a unique 64-bit long value used as an identifier for each record. This is essential for doing any kind of synchronization.
FLAGS_FIELD – This is a 32-bit int value that can be used to hold different flags for each entry. The only flag that is predefined is FLAG_SYNCHRONIZED with a value of 0x40000000.
MODIFIED_FIELD – This is a time and date value that specifies when the record was last modified.
MODIFIED_BY_FIELD – This is a 64-bit long value used to identify the database that last modified the record.
CREATED_FIELD – This is a time and date value that specifies when the record was first created.
The first two fields are the minimum required for synchronization. When a record is first created (i.e. when save() is called on it for the first time) it is assigned its own OID value. When a record is created or modified, the FLAG_SYNCHRONIZED bit is cleared.
Deleting a record that has an OID assigned will mark the record as being deleted, and may have its data erased (although its special fields will remain). It will not appear in any FoundEntries and can only be accessed using the getDeletedSince() method or the getDeletedEntry() method. The eraseEntry() method is used to remove the entry from the database completely.
The MODIFIED_FIELD and MODIFIED_BY_FIELD are updated whenever save() is called on the entry, and the CREATED_FIELD is set when the record is saved for the first time.
Note that when save() is called on a database entry, or when one of the add() or set() methods are called in the FoundEntries, then the save implementation is passed to the Database.saveEntry() method. This is the method that will set or modify all these special fields. However, during synchronization you may need to set these values yourself and then save a record without having these values altered. The Database.storeEntry() is used for this (which is also used with the FoundEntries.store() method). These methods store the DatabaseEntry as it is without changing or adding any values.
This should be done when a Database is first initialized and involves the adding of some or all of these special fields (depending on the level of synchronization you require).
A convenience method Database.enableSynchronization(int options) is provided to set up the database for synchronization. With no options specified, this method does the following:
You should not use the “_ByOID” and “_BySync” names explicitly, rather you should use the values Database.OidSortName and Database.SyncSortName. These names have “_” characters in front of them to indicate that these are not sorts that are meant to be used by the database user, but are meant for special operations instead.
If you specify the SYNC_STORE_MODIFICATION_DATE option then the method will additionally:
If you specify the SYNC_STORE_MODIFICATION_BY option then the method will additionally:
If you specify the SYNC_STORE_CREATED option then the method will additionally:
Here is an example of initializing a Database for synchronization. This example asks that the modification date and the modified by who field be used.
//===================================================================
public Database openMyDatabase(String name) throws IOException
//===================================================================
{
Database db = DatabaseManager.initializeDatabase(null,name,new Contact());
if (db != null){
db.enableSynchronization(
SYNC_STORE_MODIFICATION_DATE|SYNC_STORE_MODIFIED_BY);
db.save();
db.close();
}
return DatabaseManager.openDatabase(name,"rw");
}
This class can be used to implement a full synchronization system. In fact it is used to implement the RemoteSynchronizer described later. Synchronizer uses only the FLAG_SYNCHRONIZED bit in the FLAGS_FIELD and the OID_FIELD value to implement synchronization between two databases.
This Synchronization is a two-phase process. The first phase involves reconciling changes made on the remote database with the local database. The second phase involves reconciling changes made on the local database with the remote. The Synchronizer only has direct access to the local database (i.e. it can only search and access entries in the local database), so there are some methods that request data from the remote database that need to be implemented in order to make a full Synchronizer. This is done in the RemoteSynchronizer which uses Remote Calls to get that data.
The first phase of the Synchronization process goes like this:
Here is the simplified code for the first phase of the synchronization process:
// Get all entries in the local database, sorted by OID.
// This way we can search to find the local entry that matches
// any remote enry.
FoundEntries fe = getByOID();
int num = countRemoteUnsynchronizedEntries();
for (int i = 0; i<num; i++){
byte [] un = getRemoteUnsynchronizedEntry(i);
if (un == null) break;
DatabaseEntry de = fe.getNew();
de.decodeFrom(un);
// See if a matching local entry is unsynchronised too.
int found = getMatchingEntry(de,fe);
if (found != -1){
DatabaseEntry my = fe.get(found);
if (!isSynchronized(my)){
int resolve = resolveConflict(de,my);
if (resolve == LOCAL_TO_REMOTE){
markRemoteAsSynchronized(i);
continue;
}
}
}
addOrReplace(de,fe);
markRemoteAsSynchronized(i);
}
// Now get the OIDs of the deleted entries on the remote
// and erase them.
int deleted = countRemoteDeletedEntries();
for (int i = 0; i<deleted; i++){
long del = getRemoteDeletedEntry(i);
if (del == 0) break;
erase(del,fe);
eraseDeletedOnRemote(i);
}
The methods in green are implemented in the Synchronizer and are shown below. The methods in bold must be implemented as part of a full Synchronizer implementation.
//-------------------------------------------------------------------
private int checkSort(String sort)
//-------------------------------------------------------------------
{
int got = database.findSort(sort);
if (got == 0) throw new IllegalStateException("Database does not have the appropriate sort criteria.");
return got;
}
/**
* This gets all the unsynchronized entries, but not the deleted entries. You
* can get deleted items from the database directly.
**/
//===================================================================
public FoundEntries getUnsynchronized() throws IOException, IllegalStateException
//===================================================================
{
FoundEntries all = database.getEntries(checkSort(database.SyncSortName));
return all.getSubSet(all.findAll(new Comparer(){
public int compare(Object one,Object two){
DatabaseEntry de = (DatabaseEntry)two;
if (!de.hasField(Database.FLAGS_FIELD)) return 1;
int flags = de.getField(Database.FLAGS_FIELD,0);
if ((flags & Database.FLAG_SYNCHRONIZED) != 0) return -1;
return 0;
}
}));
}
/**
* This marks the entry at as being synchronized.
**/
//===================================================================
public void markAsSynchronized(DatabaseEntry entry)
//===================================================================
{
int value = entry.getField(Database.FLAGS_FIELD,0);
value |= Database.FLAG_SYNCHRONIZED;
entry.setField(Database.FLAGS_FIELD,value);
}
/**
* This marks the entry at the specified index as being synchronized.
**/
//===================================================================
public void markAsSynchronized(FoundEntries entries,int index) throws IOException
//===================================================================
{
DatabaseEntry de = entries.get(index);
if (de != null){
markAsSynchronized(de);
entries.store(de);
}
}
/**
* Get all the entries in the database, sorted by OID.
**/
//===================================================================
public FoundEntries getByOID() throws IOException, IllegalStateException
//===================================================================
{
return database.getEntries(checkSort(database.OidSortName));
}
/**
* This locates the entry with the specified OID. The FoundEntries object must
* have been found using the findByOID() method (i.e. using the sortByOid
* criteria).
*/
//===================================================================
public int findByOID(FoundEntries entries,long oid) throws IOException
//===================================================================
{
return entries.findFirst(new Long().set(oid));
}
/**
* Get the OID for an entry.
*/
//===================================================================
public long getOID(DatabaseEntry entry) throws IllegalStateException
//===================================================================
{
if (!entry.hasField(Database.OID_FIELD))
throw new IllegalStateException("No OID assigned.");
return entry.getField(Database.OID_FIELD,(long)0);
}
/**
* Returns the state of the synchronized flag.
*/
//===================================================================
public boolean isSynchronized(DatabaseEntry entry) throws IllegalStateException
//===================================================================
{
if (!entry.hasField(Database.FLAGS_FIELD))
throw new IllegalStateException("No Flags assigned.");
return ((entry.getField(Database.FLAGS_FIELD,(int)0) & Database.FLAG_SYNCHRONIZED) != 0);
}
//===================================================================
public int getMatchingEntry(DatabaseEntry remoteEntry,FoundEntries myEntriesSortedByOID) throws IOException, IllegalStateException
//===================================================================
{
long oid = getOID(remoteEntry);
return findByOID(myEntriesSortedByOID,oid);
}
/**
* Given a remote entry, add or replace it in this database.
**/
//===================================================================
public void addOrReplace(DatabaseEntry entry, FoundEntries myEntriesSortedByOID)
throws IOException, IllegalStateException
//===================================================================
{
int toReplace = getMatchingEntry(entry,myEntriesSortedByOID);
if (toReplace == -1){
// Was not found in my database so it must be new.
DatabaseEntry de = myEntriesSortedByOID.getNew();
de.duplicateFrom(entry);
markAsSynchronized(de);
database.storeEntry(de);
}else{
// Was found, so replace the existing entry.
DatabaseEntry de = myEntriesSortedByOID.get(toReplace);
de.duplicateFrom(entry);
markAsSynchronized(de);
myEntriesSortedByOID.store(de);
}
}
/**
* This is called after an OID marked as deleted on this database has been
* erased from the remote database. It erases it from this database too.
*/
//===================================================================
public void eraseDeleted(long oid) throws IOException
//===================================================================
{
DatabaseEntry got = database.getDeletedEntry(oid);
if (got != null) database.eraseEntry(got);
}
/**
* Given the OID of an entry in the remote database,
* erase it completely from this database.
**/
//===================================================================
public void erase(long oid,FoundEntries myEntriesSortedByOID) throws IOException
//===================================================================
{
int found = findByOID(myEntriesSortedByOID,oid);
if (found == -1) return;
myEntriesSortedByOID.erase(found);
}
Phase two of the process, sending the local unsynchronized entries to the remote database is very similar and is shown below:
FoundEntries mine = getUnsynchronized();
for (int i = 0; i<mine.size(); i++){
DatabaseEntry de = mine.get(i);
sendEntryToRemote(de);
markAsSynchronized(mine,i);
}
// Now send the deleted OIDs to the remote database.
long [] del = database.getDeletedSince(null);
for (int i = 0; i<del.length; i++){
eraseEntryOnRemote(del[i]);
eraseDeleted(del[i]);
}
<The rest of this article has not been updated for Eve as of Feb. 1, 2008>
Setting up an application for database synchronization across the desktop-mobile connection is a very simple procedure.
Note that Registry access works on Linux systems and PDAs as well as on Win32/WinCE systems. The Registry access is simulated on Linux systems using a DataStorage file called Registry.dat which is stored in the same location as the Ewe executable file.
Here is an example using our Contact example.
import ewe.database.*;
import ewe.ui.*;
import ewe.io.File;
import ewe.sys.Time;
import ewex.registry.*;
//##################################################################
public class Contact {//extends LiveObject implements HasProperties{
//##################################################################
public String lastName = "Branson";
public String firstName = "Richard";
public String company = "Virgin";
public ewe.sys.Date dob = new ewe.sys.Date();
public String _sorts = "By Name$i|lastName,firstName| By DOB$t|dob,lastName,firstName";
public String getName() {return lastName+", "+firstName;}
public String toString() {return getName();}
//
//Open the database and initialize it if necessary.
//
//===================================================================
public static Database getDatabase(String name) throws ewe.io.IOException
//===================================================================
{
Database db = DatabaseManager.initializeDatabase(null,name,new Contact());
if (db != null){
db.enableSynchronization(0);
db.save();
db.close();
}
db = DatabaseManager.openDatabase(null,name,"rw");
return db;
}
/*
This places an entry in the Registry that points to the location
of the Eve Application file that holds this class along with the
special "sync" command line switch that tells it to run in sync
mode.
This information is stored in the key:
HKEY_LOCAL_MACHINE\Software\EweSoft\Contacts
and is stored as a String value named "Sync".
This value will be looked up when the desktop synchronizer is run
in order to run the synchronizer on the mobile device.
*/
//===================================================================
private static boolean registerApplication()
//===================================================================
{
try{
File pd = File.getNewFile(File.getProgramDirectory());
File app = pd.getChild("Contacts.ewe");
RegistryKey rk = Registry.getLocalKey(
Registry.HKEY_LOCAL_MACHINE,
"Software\\EweSoft\\Contacts",true,true);
rk.setValue("Sync","\""+app.getAbsolutePath()+"\" sync");
return true;
}catch(Exception e){
return false;
}
}
/*
This is used to synchronize the contacts.
*/
//===================================================================
private static boolean synchronizeContacts()
//===================================================================
{
ProgressBarForm.display("Conctact Synchronizer",
ewe.sys.Vm.isMobile()?"Contacting desktop...":"Contacting mobile device...",null);
RemoteSynchronizer rs = null;
try{
rs = RemoteSynchronizer.synchronizeOnRemoteConnection(
//This is the database name.
"Contacts",
//This is the same key as stored in the registry.
"HKEY_LOCAL_MACHINE\\Software\\EweSoft\\Contacts\\Sync",
//This is a TimeOut value.
new ewe.sys.TimeOut(30*1000),
//These are connection options.
ewe.io.RemoteConnection.MOBILE_IS_EWE_APPLICATION|
ewe.io.RemoteConnection.MOBILE_GET_COMMAND_FROM_REGISTRY
);
}finally{
ProgressBarForm.clear();
}
ewe.sys.Handle h = rs.synchronize(
"Synchronizing Contacts","The desktop is synchronizing");
h.waitUntilStopped();
if (h.errorObject != null) return false;
return true;
}
//===================================================================
public static void main(String args[])
//===================================================================
{
ewe.sys.Vm.startEwe(args);
if (args.length == 0) try{
registerApplication();
Database db = getDatabase("Contacts");
Editor ed = DatabaseTableModel.getTestEditor(db);
ed.execute();
}catch(ewe.io.IOException e){
new ReportException(e,null,null,false).execute();
}else{
if (args[0].equals("sync"))
synchronizeContacts();
}
ewe.sys.Vm.exit(0);
}
The method RemoteSynchronizer.synchronizeOnRemoteConnection(String databaseName, String remoteCommand, TimeOut timeout, int remoteOptions) is used to make the link between the desktop synchronizer and the remote synchronizer. It works like this:
When run on the desktop it will:
Once the RemoteSynchronizer is created a call to synchronize() does the synchronization.
The example above
can be used as a simple database application complete with synchronization over
the remote connection.