The Sharedboard diagram editor examined in earlier sections uses distributed and persistent objects. Network objects allow several users to simultanously edit diagrams while persistent objects insure that the up to date diagrams are always available on persistent storage even if the diagram server crashes.
A distributed version of the diagram editor would allow several users to simultaneously see and interact with the same diagrams. This may be useful to complement a phone conversation with graphical explanations or to insure that everyone sees immediately any modifications to a diagram, to avoid the problems associated with reconciling the local changes made by several users when they take a local copy.
With network objects, it is possible to call the methods of objects which are in another process, possibly on a remote server. The stubgen tool and netobj library take care of relaying the methods and arguments between programs.
A board server maintains a table of currently opened boards. When a user requests a board already in use, he obtains a reference to that board. Thus, the board will receive editing commands from several users and must notify the other users.
INTERFACE BoardServer;
IMPORT NetObj, Thread,
Board;
TYPE T = NetObj.T OBJECT
METHODS
create (boardName: TEXT): Board.T
RAISES {Failed, NetObj.Error, Thread.Alerted};
open (boardName: TEXT): Board.T
RAISES {Failed, NetObj.Error, Thread.Alerted};
save (boardName: TEXT)
RAISES {Failed, NetObj.Error, Thread.Alerted};
close (boardName: TEXT)
RAISES {Failed, NetObj.Error, Thread.Alerted};
remove (boardName: TEXT)
RAISES {Failed, NetObj.Error, Thread.Alerted};
END;
EXCEPTION Failed (TEXT);
Once a board is obtained from the server, editing commands must be sent to it.
INTERFACE Board;
IMPORT NetObj, Thread,
Callback, Item, ClientInfo, RectR;
TYPE T = NetObj.T OBJECT
METHODS
register (cb: Callback.T): ClientInfo.T
RAISES {NetObj.Error, Thread.Alerted};
setScope (ci: ClientInfo.T; scope: RectR.T)
RAISES {NetObj.Error, Thread.Alerted};
createItems (ci: ClientInfo.T; its: Item.TArray): Item.IDArray
RAISES {NetObj.Error, Thread.Alerted};
modifyItems (ci: ClientInfo.T; its: Item.TArray; additive: BOOLEAN)
RAISES {NetObj.Error, Thread.Alerted};
deleteItems (ci: ClientInfo.T; ids: Item.IDArray)
RAISES {NetObj.Error, Thread.Alerted};
unregister (ci: ClientInfo.T)
RAISES {NetObj.Error, Thread.Alerted};
END;
Before editing a board, a new client must register with the board by providing a Callback object used to interact with the client. This Callback object will be used by the board to notify this client when other clients modify the board contents. The Callback object too is a network object.
INTERFACE Callback;
IMPORT NetObj, Thread,
Item;
TYPE T = NetObj.T OBJECT
METHODS
itemsCreated (it: Item.TArray)
RAISES {NetObj.Error, Thread.Alerted};
itemsModified (it: Item.TArray; additive: BOOLEAN)
RAISES {NetObj.Error, Thread.Alerted};
itemsDeleted (id: Item.IDArray)
RAISES {NetObj.Error, Thread.Alerted};
END;
When a client interacts with the board, it must identify itself. The client may also want to query the information stored about him in the board. Indeed, for each client, the board remembers the focus, such that the client is only notified about modified objects within its focus. When a client registers with the board, a ClientInfo network object is returned for this purpose.
INTERFACE ClientInfo;
IMPORT RectR, NetObj, Thread,
Callback;
TYPE T = NetObj.T OBJECT
METHODS
getScope (): RectR.T RAISES {NetObj.Error, Thread.Alerted};
setScope (scope: RectR.T) RAISES {NetObj.Error, Thread.Alerted};
getCallback (): Callback.T RAISES {NetObj.Error, Thread.Alerted};
END;
The simple diagram editor becomes a client editor and must connect to a BoardServer which maintains the shared boards and sends notification to clients. The procedure previously called in the diagram editor to add an item CreateItems is renamed to ItemsCreated and used when a notification to add an item is received from the board server. When a new item is added locally in the diagram editor, the add command is first sent to the board server. The CreateItems procedure thus becomes as follows.
PROCEDURE CreateItems (v: T; its: Item.TArray) =
VAR ids: Item.IDArray;
BEGIN
IF its = NIL THEN RETURN END;
TRY
(* board is a remote object in the board server. The new item is
thus first added in the shared board stored in the remote server *)
ids := v.board.createItems (v.ci, its);
LOCK v.mu DO
FOR i := FIRST (its^) TO LAST (its^) DO
its[i].id := ids[i];
EVAL v.display.put (ids[i], its[i]);
its[i].paint (v, v.focus);
END;
VBT.Sync (v);
END;
EXCEPT
NetObj.Error (atom) => Error (v, Atom.ToText (atom.head));
| Thread.Alerted => Error (v, "Thread.Alerted");
END;
END CreateItems;
The Callback network object registered in the board server is used to call the ItemsCreated and similar procedures.
INTERFACE CallbackX;
IMPORT View, Callback;
TYPE T <: Public;
Public = Callback.T OBJECT
METHODS
init (v: View.T): T;
END;
END CallbackX.
MODULE CallbackX;
IMPORT View, Item;
REVEAL T = Public BRANDED OBJECT
v: View.T;
OVERRIDES
init := Init;
itemsCreated := ItemsCreated;
itemsModified := ItemsModified;
itemsDeleted := ItemsDeleted;
END;
PROCEDURE Init (cb: T; v: View.T): T =
BEGIN
cb.v := v;
RETURN cb;
END Init;
PROCEDURE ItemsCreated (cb: T; its: Item.TArray) =
BEGIN
View.ItemsCreated (cb.v, its);
END ItemsCreated;
PROCEDURE ItemsModified (cb: T; its: Item.TArray; additive: BOOLEAN) =
BEGIN
View.ItemsModified (cb.v, its, additive);
END ItemsModified;
PROCEDURE ItemsDeleted (cb: T; ids: Item.IDArray) =
BEGIN
View.ItemsDeleted (cb.v, ids);
END ItemsDeleted;
BEGIN
END CallbackX.
Each board in the board server receives editing commands from clients and must send notification to the other clients. A separate thread is used to send the notifications in order to return the control faster to the client and to avoid deadlocks.
INTERFACE BoardX;
IMPORT Board, Item, Wr, Rd, Pickle, Thread;
TYPE T <: Public;
Public = Board.T OBJECT
METHODS
init(): T;
initT();
initS();
createItemsState(its: Item.TArray): Item.IDArray;
modifyItemsState(its: Item.TArray);
deleteItemsState(ids: Item.IDArray);
END;
PROCEDURE Busy (board: T): BOOLEAN;
MODULE BoardX;
IMPORT Thread, NetObj,
Item, ItemList, ItemTbl, AtomicItemTbl,
TextItem, RuleItem, <* NOWARN *> (* must be included *)
Callback, RectR,
ClientInfo, ClientInfoX, ClientInfoList, AtomicClientList,
NotifyRec, NotifyQueue,
Wr, Rd, Pickle;
REVEAL T = Public BRANDED OBJECT
items: AtomicItemTbl.T;
clients: AtomicClientList.T;
nq: NotifyQueue.T;
state: AtomicItemTbl.State;
OVERRIDES
init := Init;
initS := InitStable;
initT := InitTransient;
register := Register;
unregister := Unregister;
setScope := SetScope;
createItems := CreateItems;
createItemsState := CreateItemsState;
modifyItems := ModifyItems;
modifyItemsState := ModifyItemsState;
deleteItems := DeleteItems;
deleteItemsState := DeleteItemsState;
END;
CONST ExpectedItems = 1000;
(* Initialise the board *)
PROCEDURE Init (bd: T) : T =
BEGIN
InitStable(bd);
InitTransient(bd);
RETURN bd;
END Init;
PROCEDURE InitStable(bd: T) =
BEGIN
bd.state := NEW (AtomicItemTbl.State,
tbl := NEW (ItemTbl.Default).init (ExpectedItems),
id := 0);
bd.items := NEW (AtomicItemTbl.T, state := bd.state)
END InitStable;
(* A thread is forked to manage the notification of clients *)
PROCEDURE InitTransient(bd: T) =
VAR nc := NEW (NotifyClosure, bd := bd);
BEGIN
bd.clients := NEW (AtomicClientList.T);
bd.nq := NEW (NotifyQueue.T).init ();
EVAL Thread.Fork (nc);
END InitTransient;
(* Register a new client. Store its Callback network object and
return a ClientInfo network object to the client *)
PROCEDURE Register (bd: T; cb: Callback.T): ClientInfo.T =
VAR ci := NEW (ClientInfoX.T).init (cb);
BEGIN
bd.clients.add (ci);
RETURN ci;
END Register;
(* Remove a client from the list *)
PROCEDURE Unregister (bd: T; ci: ClientInfo.T) =
BEGIN
bd.clients.remove (ci);
END Unregister;
(* Change the scope of a client *)
PROCEDURE SetScope (bd: T; ci: ClientInfo.T; scope: RectR.T) =
VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Scope, doer := ci,
newScope := scope);
BEGIN
bd.nq.enq (nr);
END SetScope;
(* A new item is created, add it and queue a notification message *)
PROCEDURE CreateItems (bd: T; ci: ClientInfo.T;
its: Item.TArray): Item.IDArray =
VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Create, doer := ci,
its := its);
ids: Item.IDArray;
BEGIN
IF its = NIL THEN RETURN NIL END;
LOCK bd.items DO
ids := bd.createItemsState(its);
END;
bd.nq.enq (nr);
RETURN ids;
END CreateItems;
PROCEDURE CreateItemsState (bd: T; its: Item.TArray): Item.IDArray =
VAR ids: Item.IDArray;
BEGIN
ids := NEW (Item.IDArray, NUMBER (its^));
FOR i := FIRST (its^) TO LAST (its^) DO
INC (bd.items.state.id);
ids[i] := bd.items.state.id;
its[i].id := ids[i];
EVAL bd.items.state.tbl.put (its[i].id, its[i]);
END;
RETURN ids;
END CreateItemsState;
(* An item is modified, do it and queue a notification message *)
PROCEDURE ModifyItems (bd: T; ci: ClientInfo.T;
its: Item.TArray; additive: BOOLEAN) =
VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Modify, doer := ci,
its := its, additive := additive);
BEGIN
IF its = NIL THEN RETURN END;
LOCK bd.items DO
bd.modifyItemsState(its);
END;
bd.nq.enq (nr);
END ModifyItems;
PROCEDURE ModifyItemsState (bd: T; its: Item.TArray) =
VAR old: Item.T;
BEGIN
FOR i := FIRST (its^) TO LAST (its^) DO
IF its[i] = NIL OR NOT bd.items.state.tbl.get (its[i].id, old) THEN
its[i] := NIL;
ELSE
EVAL bd.items.state.tbl.put (its[i].id, its[i]);
END;
END;
END ModifyItemsState;
(* An item is deleted, do it and queue a notification message *)
PROCEDURE DeleteItems (bd: T; ci: ClientInfo.T; ids: Item.IDArray) =
VAR nr := NEW (NotifyRec.T, code := NotifyRec.Code.Delete, doer := ci,
ids := ids);
BEGIN
IF ids = NIL THEN RETURN END;
LOCK bd.items DO
bd.deleteItemsState(ids);
END;
bd.nq.enq (nr);
END DeleteItems;
PROCEDURE DeleteItemsState (bd: T; ids: Item.IDArray) =
VAR old: Item.T;
BEGIN
FOR i := FIRST (ids^) TO LAST (ids^) DO
EVAL bd.items.state.tbl.delete (ids[i], old);
END;
END DeleteItemsState;
(* The notification thread awaits notification messages and
sends the notification to clients *)
TYPE NotifyClosure = Thread.Closure OBJECT
bd: T;
OVERRIDES
apply := NotifyLoop;
END;
PROCEDURE NotifyLoop (nc: NotifyClosure): REFANY =
VAR nr: NotifyRec.T;
BEGIN
LOOP
nr := nc.bd.nq.deq ();
Notify (nc.bd, nr);
END;
END NotifyLoop;
PROCEDURE Notify (bd: T; nr: NotifyRec.T) =
BEGIN
CASE nr.code OF
(* If a message to a client does not get through (Netobj.Error),
the client is simply unregistered and looses the connection *)
(* The scope of a client was changed, send to this client only
all the items in the new scope. When a client first connects,
its scope (focus) is empty and later set to something else,
it then automatically receives all the items in its scope. *)
NotifyRec.Code.Scope =>
LOCK bd.items DO
VAR ir := bd.items.state.tbl.iterate ();
id: Item.ID;
it: Item.T;
il: ItemList.T := NIL;
oldScope := nr.doer.getScope(); <*NOWARN*>
BEGIN
WHILE ir.next (id, it) DO
IF RectR.Overlap (it.box, nr.newScope)
THEN
il := ItemList.Cons (it, il);
END;
END;
VAR its := NEW (Item.TArray, ItemList.Length (il));
i := 0;
BEGIN
WHILE il # NIL DO
its [i] := il.head;
il := il.tail;
INC (i);
END;
TRY
nr.doer.getCallback().itemsCreated (its);
nr.doer.setScope (nr.newScope);
EXCEPT
NetObj.Error, Thread.Alerted => Unregister (bd, nr.doer);
END;
END;
END;
END;
(* All clients except the one from which the item creation originated
are notified *)
| NotifyRec.Code.Create =>
VAR deadClients: ClientInfoList.T := NIL; BEGIN
LOCK bd.clients DO
VAR clients := bd.clients.list; BEGIN
WHILE clients # NIL DO
IF clients.head # nr.doer THEN
TRY
clients.head.getCallback().itemsCreated (nr.its);
EXCEPT
NetObj.Error, Thread.Alerted =>
deadClients := ClientInfoList.Cons (clients.head,
deadClients);
END;
END;
clients := clients.tail;
END;
END;
END;
(* must remove deadClients after unlocking bd.clients *)
WHILE deadClients # NIL DO
Unregister (bd, deadClients.head);
deadClients := deadClients.tail;
END;
END;
(* All clients except the one from which the item modification
originated are notified *)
| NotifyRec.Code.Modify =>
VAR deadClients: ClientInfoList.T := NIL; BEGIN
LOCK bd.clients DO
VAR clients := bd.clients.list; BEGIN
WHILE clients # NIL DO
IF clients.head # nr.doer THEN
TRY
clients.head.getCallback().itemsModified (nr.its,
nr.additive);
EXCEPT
NetObj.Error, Thread.Alerted =>
deadClients := ClientInfoList.Cons (clients.head,
deadClients);
END;
END;
clients := clients.tail;
END;
END;
END;
(* must remove deadClients after unlocking bd.clients *)
WHILE deadClients # NIL DO
Unregister (bd, deadClients.head);
deadClients := deadClients.tail;
END;
END;
(* All clients except the one from which the item deletion
originated are notified *)
| NotifyRec.Code.Delete =>
VAR deadClients: ClientInfoList.T := NIL; BEGIN
LOCK bd.clients DO
VAR clients := bd.clients.list; BEGIN
WHILE clients # NIL DO
IF clients.head # nr.doer THEN
TRY
clients.head.getCallback().itemsDeleted (nr.ids);
EXCEPT
NetObj.Error, Thread.Alerted =>
deadClients := ClientInfoList.Cons (clients.head,
deadClients);
END;
END;
clients := clients.tail;
END;
END;
END;
(* must remove deadClients after unlocking bd.clients *)
WHILE deadClients # NIL DO
Unregister (bd, deadClients.head);
deadClients := deadClients.tail;
END;
END;
END;
END Notify;
(* Any client remaining? *)
PROCEDURE Busy (bd: T): BOOLEAN =
BEGIN
RETURN bd.clients.list # NIL;
END Busy;
BEGIN
END BoardX.
The notification records simply store the originating client, the operation and the associated information.
INTERFACE NotifyRec;
IMPORT RectR,
Item, ClientInfo;
TYPE T = REF RECORD
code: Code;
doer: ClientInfo.T;
ids: Item.IDArray := NIL;
its: Item.TArray := NIL;
newScope := RectR.Empty;
additive: BOOLEAN;
END;
Code = {Scope, Create, Modify, Delete};
The notify queue must store notification records and synchronize queing and dequeing operations. Indeed, several clients may send editing commands simultaneously and the network objects library may allocate several threads to handle these requests. Proper locking insures that only one thread at a time queues a notification record. Only the notification thread removes notification records from the queue but locking is also required to guard against the other threads that queue records.
INTERFACE NotifyQueue;
IMPORT NotifyRec;
TYPE T <: Public;
Public = Private OBJECT METHODS
init (): T;
enq (nr: NotifyRec.T);
deq (): NotifyRec.T;
END;
Private <: ROOT;
MODULE NotifyQueue;
IMPORT Thread,
NotifyRec, NotifyRecList;
REVEAL Private = MUTEX BRANDED "NotifyQueueP" OBJECT END;
T = Public BRANDED "NotifyQueue" OBJECT
list: NotifyRecList.T := NIL;
nonEmpty: Thread.Condition;
OVERRIDES
init := Init;
enq := Enq;
deq := Deq;
END;
PROCEDURE Init (nq: T): T =
BEGIN
nq.nonEmpty := NEW (Thread.Condition);
RETURN nq;
END Init;
(* Lock before adding and signal that the queue is not empty any more *)
PROCEDURE Enq (nq: T; nr: NotifyRec.T) =
BEGIN
LOCK nq DO
nq.list := NotifyRecList.AppendD (nq.list, NotifyRecList.List1 (nr));
Thread.Signal (nq.nonEmpty);
END;
END Enq;
(* If the queue is empty, the notification thread has nothing better to
do than wait. *)
PROCEDURE Deq (nq: T): NotifyRec.T =
VAR nr: NotifyRec.T;
BEGIN
LOCK nq DO
WHILE nq.list = NIL DO Thread.Wait (nq, nq.nonEmpty) END;
nr := nq.list.head;
nq.list := nq.list.tail;
RETURN nr;
END;
END Deq;
BEGIN
END NotifyQueue.
The board server maintains a table of the boards currently in use. Being a network object, it is accessed by clients to connect to boards.
MODULE BoardServerX;
IMPORT Board, BoardServer, StableBoardXTbl,
BoardX, StableBoardX, StableError;
REVEAL T = Public BRANDED OBJECT
mu: MUTEX;
boards: StableBoardXTbl.Default;
OVERRIDES
init := Init;
create := Create;
open := Open;
save := Save;
close := Close;
remove := Remove;
END;
CONST ExpectedBoards = 10;
PROCEDURE Init (bs: T): T =
BEGIN
bs.mu := NEW (MUTEX);
bs.boards := NEW (StableBoardXTbl.Default).init (ExpectedBoards);
RETURN bs;
END Init;
PROCEDURE Create (bs: T; boardName: TEXT): Board.T
RAISES {BoardServer.Failed} =
BEGIN
RETURN Open(bs, boardName);
END Create;
(* Find a board in the table of currently used boards or add it in
the table if not there *)
PROCEDURE Open (bs: T; boardName: TEXT): Board.T
RAISES {BoardServer.Failed} =
VAR board: StableBoardX.T;
recovered: BOOLEAN;
BEGIN
TRY
LOCK bs.mu DO
IF bs.boards.get (boardName, board) THEN (* in-memory *)
RETURN board;
ELSE (* load board *)
board := NEW (StableBoardX.T).init (boardName, recovered);
IF (NOT recovered) THEN
board.initS();
END;
board.initT();
EVAL bs.boards.put (boardName, board);
RETURN board;
END;
END;
EXCEPT
StableError.E =>
RAISE BoardServer.Failed ("Could not open " & boardName);
END;
END Open;
(* Write the board to disk *)
PROCEDURE Save (bs: T; boardName: TEXT)
RAISES {BoardServer.Failed} =
VAR board: StableBoardX.T;
BEGIN
TRY
LOCK bs.mu DO
IF bs.boards.get (boardName, board) THEN
StableBoardX.Checkpoint(board);
ELSE
RAISE BoardServer.Failed ("Board not loaded");
END;
END;
EXCEPT
StableError.E =>
RAISE BoardServer.Failed ("Error saving board " & boardName);
END;
END Save;
(* If no client remains for the closed board, save it and remove it
from the table *)
PROCEDURE Close (bs: T; boardName: TEXT)
RAISES {BoardServer.Failed} =
VAR board: StableBoardX.T;
BEGIN
TRY
LOCK bs.mu DO
IF bs.boards.get (boardName, board) THEN
IF NOT BoardX.Busy (board) THEN
EVAL bs.boards.delete (boardName, board);
StableBoardX.Checkpoint(board);
BoardX.Quit (board);
END;
ELSE
RAISE BoardServer.Failed ("Board not loaded");
END;
END;
EXCEPT
StableError.E =>
RAISE BoardServer.Failed ("Error saving board " & boardName);
END;
END Close;
(* If no client remains for that board, remove it altogether *)
PROCEDURE Remove (bs: T; boardName: TEXT)
RAISES {BoardServer.Failed} =
VAR board: StableBoardX.T;
BEGIN
TRY
LOCK bs.mu DO
IF bs.boards.get (boardName, board) THEN
IF BoardX.Busy (board) THEN
RAISE BoardServer.Failed ("Board is busy")
END;
EVAL bs.boards.delete (boardName, board);
board.dispose();
END
END
EXCEPT
StableError.E =>
RAISE BoardServer.Failed ("Problems deleting " & boardName);
END;
END Remove;
BEGIN
END BoardServerX.
The main program of the board server simply exports the BoardServer network object to the Network Object Daemon. The clients will obtain the BoardServer object from the daemon. The BoardServer object will then be used to obtain board objects.
MODULE Server EXPORTS Main;
IMPORT NetObj, Thread, Err,
BoardServerX;
VAR bs := NEW (BoardServerX.T).init ();
BEGIN
TRY
NetObj.Export ("BoardServer", bs);
EXCEPT
NetObj.Error => Err.Print ("Please start netobjd and retry.\n");
END;
(* Do nothing, the action takes place when methods of the BoardServer
object are invoked by remote clients. *)
LOOP Thread.Pause (10.0d0); END;
END Server.
In the diagram editor example shown in earlier sections, little was said about reading or writing boards (diagrams) from disk. Since information about object types is available at run time, procedures exist to read or write graphs of objects. These procedures are found in the Pickle library. Such procedures obviate the need for defining a file format and writing input and output procedures.
PROCEDURE PutState(bd: T; wr: Wr.T)
RAISES {Pickle.Error, Wr.Failure, Thread.Alerted} =
BEGIN
Pickle.Write(wr, bd);
END PutState;
PROCEDURE GetState(bd: T; rd: Rd.T): T
RAISES {Pickle.Error, Rd.Failure, Rd.EndOfFile, Thread.Alerted} =
BEGIN
bd := Pickle.Read(rd);
RETURN bd;
END GetState;
In practice it is often needed to split the object state into a persistent portion, which should be saved to disk, and a transient portion containing mutexes, list or currently registered clients...
Migrating objects between the disk and memory, using a library such as Pickle, is a first step towards offering object oriented services similar to traditional databases. The next step is insuring that all modifications to the persistent objects are automatically propagated to disk even if the program or computer suddenly stops. Recovering from disk failures is another problem usually dealt with through disk mirroring or frequent backups to tape archives.
The SmallDB and Stable libraries, along with the StableGen tool may be used to provide persistent objects with little efforts. To add persistence to an object type, its declaration is passed to StableGen which generates a new derived type with the following methods and procedures added:
Furthermore, all the methods which affect the object state are overridden to first write the method name and arguments to a log, then call the original method. Thus, when a persistent object is initialized after a crash, its state is restored from the last checkpoint and all the method calls stored in the log are redone to get back to the exact state prior to the crash.
The BoardX.i3 interface describes the methods for accessing boards. Pragmas are added to identify the type for which a stable derived type is wanted and the methods that modify the object state, and thus which need to be recorded in the change log.
INTERFACE BoardX;
IMPORT Board, Item, Wr, Rd, Pickle, Thread;
TYPE T <: Public;
Public = Board.T OBJECT
METHODS
init(): T;
initT();
initS();
createItemsState(its: Item.TArray): Item.IDArray;
modifyItemsState(its: Item.TArray);
deleteItemsState(ids: Item.IDArray);
END;
<*PRAGMA STABLE *>
<*STABLE UPDATE METHODS initS, createItemsState, modifyItemsState,
deleteItemsState *>
The new stable derived type is created by stablegen and is named StableBoardX.T. It is used to initialise board objects from their persistent state, if it exists, and to periodically write a checkpoint of the state of the board. The frequency at which checkpoints are written is a compromise between the time taken to write the checkpoint, and the space occupied by the log of changes since the last checkpoint, as well as the recovery time which is proportional to the log length. In this case, the checkpointing criterion is simple. Every time the user issues a save command or closes a board, a new checkpoint is written and the change log emptied.
When a board is opened or created by the board server, the following actions are sufficient to create a board and recover its persistent state, if it exists.
(* The board name identifies the checkpoint and log files on disk *)
board := NEW (StableBoardX.T).init (boardName, recovered);
(* This is a newly created board, there is no state for it yet *)
IF (NOT recovered) THEN
board.initS();
END;
board.initT();
When the board is saved or closed, a new checkpoint is written as follows.
TRY
StableBoardX.Checkpoint(board);
EXCEPT
StableError.E =>
RAISE BoardServer.Failed ("Error saving board " & boardName);
END;
If a board should no longer be persistent, when the remove command is issued by the user, the dispose method is called and the checkpoint and log files associated with the board are deleted.
TRY
board.dispose();
EXCEPT
StableError.E =>
RAISE BoardServer.Failed ("Problems deleting " & boardName);
END;
For small databases that fit into memory, indexes are easily built using the Table (hash table) and SortedTable (sorted tree) interfaces. In the diagram example, indexes for the X and Y upper left coordinates of the items could be implemented using sorted tables that map the coordinate (REAL) to the item identifier (CARDINAL). These indexes are added to the Board object declaration.
REVEAL T = Public BRANDED OBJECT
items: AtomicItemTbl.T;
indexX: RealItemIDSortedTable.T;
indexY: RealItemIDSortedTable.T;
Then, each time new items are created in CreateItemsState, they must be entered in the indexes.
(* insert the item into the board items table *)
WITH item = its[i] DO
EVAL bd.items.state.tbl.put (item.id, item);
EVAL bd.indexX.put(item.box.west,item.id);
EVAL bd.indexY.put(item.box.north,item.id);
END;
Similarly, when existing items are modified, the indexes must be updated in ModifyItemsState.
WITH item = its[i] DO
(* Retrieve the existing item and remove it from the indexes *)
EVAL bd.items.state.tbl.get (item.id, oldItem)
EVAL bd.indexX.delete(oldItem.box.west,it);
EVAL bd.indexY.delete(oldItem.box.north,it);
(* Install the new, modified, item in the table and indexes *)
EVAL bd.items.state.tbl.put (item.id, item);
EVAL bd.indexX.put(item.box.west,item.id);
EVAL bd.indexY.put(item.box.north,item.id);
END;
Finally, when an item is deleted, the index entries are removed in DeleteItemsState.
WITH item = its[i] DO
(* Retrieve the existing item and remove it from the indexes *)
EVAL bd.items.state.tbl.get (item.id, oldItem)
EVAL bd.indexX.delete(oldItem.box.west,it);
EVAL bd.indexY.delete(oldItem.box.north,it);
END;
The Queries on these indexes would return the sorted sequence of items in a speficied coordinate range.
PROCEDURE FindItems(tbl: RealItemIDSortedTable.T; min, max: REAL):
IDSequence.T =
VAR
seq := NEW(IDSequence.T).init();
iterator := tbl.iterateOrdered();
cond := TRUE;
key: REAL;
val: Item.ID;
BEGIN
cond := iterator.seek(min);
LOOP
IF iterator.next(key,val) THEN
IF key <= max THEN seq.addhi(val);
ELSE EXIT;
END;
ELSE EXIT;
END;
END;
SortIDSequence(seq);
RETURN seq;
END FindItems;
The intersection and union of such sequences may then be used to construct more complex queries.
PROCEDURE Intersect(seq1, seq2: IDSequence.T): IDSequence.T =
VAR
outSeq: NEW(IDSequence.T).init();
i1 := 0;
i2 := 0;
end1 := seq1.size();
end2 := seq2.size();
BEGIN
WHILE i1 < end1 AND i2 < end2 DO
WITH id1 = seq1.get(i1), id2 = seq2.get(i2) DO
IF id1 < id2 THEN
INC(i1);
ELSIF id2 < id1 THEN
INC(i2);
ELSE
outSeq.addhi(id1);
INC(i1);
INC(i2);
END;
END;
END;
END Intersect;
PROCEDURE Union(seq1, seq2: IDSequence.T): IDSequence.T =
VAR
outSeq: NEW(IDSequence.T).init();
i1 := 0;
i2 := 0;
end1 := seq1.size();
end2 := seq2.size();
BEGIN
WHILE i1 < end1 AND i2 < end2 DO
WITH id1 = seq1.get(i1), id2 = seq2.get(i2) DO
IF id1 < id2 THEN
outSeq.addhi(id1);
INC(i1);
ELSIF id2 < id1
outSeq.addhi(id2);
INC(i2);
ELSE
outSeq.addhi(id1);
INC(i1);
INC(i2);
END;
END;
END;
WHILE i1 < end1 DO
outSeq.addhi(seq1.get(i1));
INC(i1);
END;
WHILE i2 < end2 DO
outSeq.addhi(seq2.get(i2));
INC(i2);
END;
END Union;
Database systems often offer a specialized query language. Queries are parsed and optimized in order to translate them into efficient low level database operations. For example, when looking for all items with an X coordinate between 0.1 and 0.11 and a Y coordinate between 0 and 1, at least three possible paths may lead to the desired result.
If the database contains 1000 items uniformly distributed in the square defined by X and Y between 0 and 1, there will be approximately 10 items with X between 0.10 and 0.11 and 1000 items with Y between 0 and 1. Thus, the first solution finds a list of 10 and a list of 1000 and performs the intersection. The third solution is no better since it finds a lists of 1000 and checks each item for its X coordinate. The second solution in this case is clearly superior since it obtains a list of 10 items through the X index and checks the Y coordinate of each.
Traditional sophisticated databases include support for atomic transactions. A transaction may be initiated, then a sequence of updating commands are issued and finally the transaction is ended with a commit or abort command. This way, the sequence of updating commands becomes atomic, they are all performed together if the commit is successfull or none of them is performed if the database crashes, the commit is not successfull or abort is issued.
In stable persistent objects, methods are atomic (logged atomically) but there is no explicit support for transactions. The same effect however may be obtained in different ways. For instance, all updating actions which need to be performed together may be grouped in a single atomic method.
Another possibility is to keep an undo list. The undo list starts recording when the begin_transaction method is called and is emptied after a successfull commit. However, when recovering from a crash or when the transaction is aborted the undo list would be used to recover the state prior to the begin_transaction.
In the SmallDB and Stable libraries, no special mechanism is required to fit all the database in memory. The database must fit in virtual memory (less than 100MB or so) and the usual operating system swapping mechanisms are used to control which pages remain in main memory and which pages are left on disk. The performance is excellent since all the database is already converted to the in memory format, most queries are handled directly in main memory, and a single disk write is needed for each database update.
For larger databases (hundreds of megabytes to tens of gigabytes), however, special implementation tricks must be used. The loading, caching and unloading of database pages, from disk files to virtual memory, is managed directly by the database server. Objects are laid out on pages in such a way that little or no translation is required when they are loaded or unloaded (no pointers are used). The placement of objects on pages is optimized to reduce the number of pages to load for typical queries. For example, indexes are organized as balanced trees where each tree node fits exactly on a page.
In many cases, the database server accesses directly the raw disk device to avoid going through the operating system input-output buffer cache and to optimize the layout on disk. The result is some performance increase for very large databases at a huge cost in implementation complexity and non portability.