Keeping string contents in memory across requests?
let me first tell you a bit of the background to my recent problem.
Some months ago, I wrote a series of PHP scripts, that are based around an SQLite database. One part of the scripts is used to generate new or fetch existing external data, which is then saved in the db-file. The other half of the scripts later extracts this data and removes it from the database after it has been used.
The problem stems from the new level of use the scripts have seen and the way these scripts are designed: if data is fetched from an external source, it always happens in smaller pieces (of 5-100 kilobyte) per single request. Data-input tasks are split up in most cases, which means, that a single task can generate a few dozen HTTP-requests and disk-writes or more. The read/delete part of the equation have proven to not be an issue.
For technical reasons - I'm not root on the server this runs on and have no physical access to the hardware - I can not use a different SQL database with a server process and load the whole database into the RAM, put in an SSD for better performance or maybe even create a RAM-disk and move the db-file there. Since the process of fetching external data will always be fragmented, this creates a problem. When too many scripts are both writing and reading/deleting from the SQLite database file, the CPU load spikes due to the high number of disk-writes and the hard-disk fails to keep up with the demand created by the many disk-seeks and -writes.
Now my question is: can I somehow keep a few strings (to the total amount of a few megabytes at a time) in the memory for a short while and later access them from a different script instance? At first, I thought about using sessions, but as far as I know, they will also be saved to disk immediately.
If an in-memory solution exists, I would be able to cache everything until the last fragmented database entry and write them in one go at the end. This would reduce the number of disk-writes by a few orders of magnitude and solve my problem, because the current level of activity will not rise again.
If it's not possible, a new server is needed and I will have free choice of the different solutions I already outlined earlier anyway.
Thanks for your help,