My JavaScript book is out! Don't miss the opportunity to upgrade your beginner or average dev skills.

Friday, June 08, 2012

Asynchronous Storage For All Browsers

I have finally implemented and successfully tested the IndexedDB fallback for Firefox so that now every browser, old or new, should be able to use this interface borrowed from localStorage API but made asynchronous.

Asynchronous Key/Value Pairs

The main purpose of this asyncStorage API is to store large amount of data as string, including base64 version of images or other files.
As it is, usually, values are the bottleneck, RAM consumption speaking, while keys are rarely such big problem.
However, while keys are retrieved asynchronously and in a non-blocking way, but kept in memory, respective values are always retrieved asynchronously in order to do not fill the available amount of RAM for our Web Application.

Database Creation/Connection

Nothing more than ...

asyncStorage.create("my_db_name", function (db, numberOfItems) {
// do stuff with the asyncStorage
});


Storing An Item

As it is for localStorage, but async

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.setItem("a", "first entry", function () {
// done, item stored
});
});

Please note that if the item was there already, it's simply replaced with the new value.

Getting An Item

As it is for localStorage, but async

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.getItem("a", function (value) {
// done, value retrieved
});
});

Please note that if the item was not previously stored, the returned value will be exactly null as it is for localStorage.

Removing An Item

As it is for localStorage, but async

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.removeItem("a", function () {
// done, no key a is present anymore
// so length is decreased already here
// and db.get("a") will send a null value
});
});


Removing All Items

As it is for localStorage, but async, and considering that only values in the specified database name will be erased, rather than all of them.

asyncStorage.create("my_db_name", function (db, numberOfItems) {
db.clear(function () {
// done, database "my_db_name" is now empty
});
});


Getting All Items Keys

If this is really what you need to do, bear in mind the API is the same used in the localStorage where indeed there's no way to retrieve all keys if not doing something like:

for (var
keys = [],
i = db.length;
i--;
) {
keys[i] = db.key(i);
}


Getting All Items

Well, the thing here is that you can store an entire object through JSON so if you need to save and get back everything, it's kinda pointless to store different keys, just use one.
However, this is how I would do this task:

for (var
object = {},
complete = function () {
alert("Done, all items in the object");
},
ongetitem = function (value, key) {
object[key] = value;
if (!--j) complete();
},
i = db.length,
j = i;
i--;
) {
db.getItem(db.key(i), ongetitem);
}


On Github, Of Course

You can find full API description in this repository. Please forgive me if the name was initially db.js, I believe the new one, asyncStorage.js, is much more appropriate (also another guy created a script called db.js so ... well, I have avoided conflicts with that library).

That's Pretty Much It

And I hope you'll start using this API to avoid blocking mobile and desktop browsers when you store a lot of data ;)

6 comments:

Aadaam said...

Two remarks:

1) function (db, numberOfItems) -> shouldn't this be maxNumberOfItems instead?

2) db.setItem("a", "first entry", function () {
// done, item stored
});

Regardless of maxNumberOfItems, perhaps this function should take a parameter, in case anything bad happens? (like, the developer didn't respect numberOfItems)

As I see the implementation, errorCallback isn't called ever by the cookie on set. Question is, are success and error worth two callbacks or it is better to make it one nodejs style?

BTW, the variable readTransaction just magically appears in WebSQL, (it's defined in indexedDB), I don't know if it's a bug which accidentally works (by having the right order in build), or it was deliberate..

Andrea Giammarchi said...

Aadaam, numberOfItems is simply the equivalent of db.length, it might be useful to know if there are items stored already or none of them (e.g. very first time)

However, I find the second argument redundant so this might change soon or be removed.

If anything bad happens you can send the second callback, as it is for Web SQL Database.
In the git repository you have full specs for the API, in this case would be
db.setItem(key, value, callback, errorback);
where both are optional in case you just want to store something without actions after.

The cookie version is really IE less than 8 story, I need more time to test there properly after implementing failures in the test to cover all error cases too.
This is in the TODO list at the end of the git repository.

readTransaction is defined the first time in asyncStorage.js, then IndexedDB borrows some variable reference/type just to make minifiers life easier having slightly better ratio over shrinking.

It looks weird, but it does the job right now and yes, the order of definition is relevant for those 3 variables (executeSql too)

Thanks for your comment in any case :)

Unknown said...

Your test page http://www.3site.eu/db/test/ fails for IE9.

Andrea Giammarchi said...

thanks, I have fixed a typo in the code for localStorage and cookies, now it works ;-)

Klaitos said...

Great article, but is it safe to use it ? like creating a cart engine (litte e-commerce site) to store data through a multiple form page ? Or have you got an exemple where to use this API ?
Thank !

Andrea Giammarchi said...

Klaitos, whenever it makes sense to you to store big amount of serialized data (through JSON, as example) you can use this API without problems ... tests are green, if you find cases where it fails, well, feel free to poke me ;-)