You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
112 lines
5.1 KiB
112 lines
5.1 KiB
# file-entry-cache |
|
> Super simple cache for file metadata, useful for process that work o a given series of files |
|
> and that only need to repeat the job on the changed ones since the previous run of the process — Edit |
|
|
|
[](https://npmjs.org/package/file-entry-cache) |
|
[](https://travis-ci.org/royriojas/file-entry-cache) |
|
|
|
## install |
|
|
|
```bash |
|
npm i --save file-entry-cache |
|
``` |
|
|
|
## Usage |
|
|
|
The module exposes two functions `create` and `createFromFile`. |
|
|
|
## `create(cacheName, [directory, useCheckSum])` |
|
- **cacheName**: the name of the cache to be created |
|
- **directory**: Optional the directory to load the cache from |
|
- **usecheckSum**: Whether to use md5 checksum to verify if file changed. If false the default will be to use the mtime and size of the file. |
|
|
|
## `createFromFile(pathToCache, [useCheckSum])` |
|
- **pathToCache**: the path to the cache file (this combines the cache name and directory) |
|
- **useCheckSum**: Whether to use md5 checksum to verify if file changed. If false the default will be to use the mtime and size of the file. |
|
|
|
```js |
|
// loads the cache, if one does not exists for the given |
|
// Id a new one will be prepared to be created |
|
var fileEntryCache = require('file-entry-cache'); |
|
|
|
var cache = fileEntryCache.create('testCache'); |
|
|
|
var files = expand('../fixtures/*.txt'); |
|
|
|
// the first time this method is called, will return all the files |
|
var oFiles = cache.getUpdatedFiles(files); |
|
|
|
// this will persist this to disk checking each file stats and |
|
// updating the meta attributes `size` and `mtime`. |
|
// custom fields could also be added to the meta object and will be persisted |
|
// in order to retrieve them later |
|
cache.reconcile(); |
|
|
|
// use this if you want the non visited file entries to be kept in the cache |
|
// for more than one execution |
|
// |
|
// cache.reconcile( true /* noPrune */) |
|
|
|
// on a second run |
|
var cache2 = fileEntryCache.create('testCache'); |
|
|
|
// will return now only the files that were modified or none |
|
// if no files were modified previous to the execution of this function |
|
var oFiles = cache.getUpdatedFiles(files); |
|
|
|
// if you want to prevent a file from being considered non modified |
|
// something useful if a file failed some sort of validation |
|
// you can then remove the entry from the cache doing |
|
cache.removeEntry('path/to/file'); // path to file should be the same path of the file received on `getUpdatedFiles` |
|
// that will effectively make the file to appear again as modified until the validation is passed. In that |
|
// case you should not remove it from the cache |
|
|
|
// if you need all the files, so you can determine what to do with the changed ones |
|
// you can call |
|
var oFiles = cache.normalizeEntries(files); |
|
|
|
// oFiles will be an array of objects like the following |
|
entry = { |
|
key: 'some/name/file', the path to the file |
|
changed: true, // if the file was changed since previous run |
|
meta: { |
|
size: 3242, // the size of the file |
|
mtime: 231231231, // the modification time of the file |
|
data: {} // some extra field stored for this file (useful to save the result of a transformation on the file |
|
} |
|
} |
|
|
|
``` |
|
|
|
## Motivation for this module |
|
|
|
I needed a super simple and dumb **in-memory cache** with optional disk persistence (write-back cache) in order to make |
|
a script that will beautify files with `esformatter` to execute only on the files that were changed since the last run. |
|
|
|
In doing so the process of beautifying files was reduced from several seconds to a small fraction of a second. |
|
|
|
This module uses [flat-cache](https://www.npmjs.com/package/flat-cache) a super simple `key/value` cache storage with |
|
optional file persistance. |
|
|
|
The main idea is to read the files when the task begins, apply the transforms required, and if the process succeed, |
|
then store the new state of the files. The next time this module request for `getChangedFiles` will return only |
|
the files that were modified. Making the process to end faster. |
|
|
|
This module could also be used by processes that modify the files applying a transform, in that case the result of the |
|
transform could be stored in the `meta` field, of the entries. Anything added to the meta field will be persisted. |
|
Those processes won't need to call `getChangedFiles` they will instead call `normalizeEntries` that will return the |
|
entries with a `changed` field that can be used to determine if the file was changed or not. If it was not changed |
|
the transformed stored data could be used instead of actually applying the transformation, saving time in case of only |
|
a few files changed. |
|
|
|
In the worst case scenario all the files will be processed. In the best case scenario only a few of them will be processed. |
|
|
|
## Important notes |
|
- The values set on the meta attribute of the entries should be `stringify-able` ones if possible, flat-cache uses `circular-json` to try to persist circular structures, but this should be considered experimental. The best results are always obtained with non circular values |
|
- All the changes to the cache state are done to memory first and only persisted after reconcile. |
|
|
|
## License |
|
|
|
MIT |
|
|
|
|
|
|