Confirm
595
edits
Changes
m
→How to Store Your Data
* If you need to store a small amount of data (less than 1MB), you should use JSON files, and do all your I/O off the main thread. Simple solutions with linear worst-case performance are ideal.
** If you're working with larger amounts of data (roughly on the order of 1MB), you should compress the JSON data with Snappy or lz4 before writing it to disk
** You can use OS.File's [https://developer.mozilla.org/en-US/docs/JavaScript_OS.File/OS.File_for_the_main_thread#OS.File.writeAtomic%28%29 writeAtomic()] method. More information on writeAtomic [http://dutherenverseauborddelatable.wordpress.com/2014/02/05/is-my-data-on-the-disk-safety-properties-of-os-file-writeatomic/ in this blog post]
** If your workload involves a lot of strings, don't use SQLite. Store your data in external files.
** If your workload involves a lot of blobs, don't use SQLite. Store your data in external files.
* If you have a large dataset, but you don't need to run complex queries over it, evaluate a JSON files structure on disk
* If you must read and write most of your data every time, evaluate using a JSON file
* For larger datasets or when SQL is absolutely necessary, use SQLite.
** Make sure you understand [https://sqlite.org/docs.html how SQLite works], carefully design your schemas and then profile your implementation.
