Statement Deletion API
- Aodhan Wade
- Toran Sharma
- James Mullaney
STATEMENT DELETION HTTP INTERFACES
ADDED IN V3.11.0
By default, Learning Locker gives you the ability to delete statements via the API (this can be disabled via the ENABLE_STATEMENT_DELETION flag).
Statements may be deleted individually, using the record’s _id, or in bulk via a batch delete method.
AUTHORIZATION
When using this interface, you must additionally supply your Basic Auth details with each request in the Authorization header. Your Basic Auth details can be found under Settings > Clients.
The deletion APIs require that the “Delete statements” ('statements/delete') scope is enabled on the client. If a client also has a store (lrs_id) attached, this will be used to further filter down deletions only to this store.
SINGLE STATEMENT DELETION
This leverages the existing REST API for statements.
DELETE http://www.example.org/api/v2/statement/111aaa1111a111111aa11112
Authorization: Basic YOUR_BASIC_AUTH
A request like the one above, will respond with a 204 response like the one below.
HTTP/1.1 204 NO CONTENT
The statement is deleted immediately and the 204 represents a receipt of the deletion event succeeding in the database.
BATCH STATEMENT DELETION
These endpoints allow you to send in deletion jobs to be processed, as well as stop a batch deletion job.
As the filter passed in may apply to a large amount of data, the batch delete job is split out into batches, each deleting up to 1000 records at a time. Each successive batch will trigger another deletion job to the worker, until no more statements exist in the database matching that filter.
INITIALISING A BATCH DELETION
Sending a POST with a JSON body holding the required deletion filter to the following endpoint will create a job to remove all data matching that filter from the respective organisation (or store) that the client is attached to.
e.g. Deletes all completions in the client’s organisation or store
POST http://www.example.org/api/v2/batchdelete/initialise
Authorization: Basic YOUR_BASIC_AUTH
Content-Type: application/json; charset=utf-8
{
"filter": {
"statement.verb.id": "http://adlnet.gov/expapi/verbs/completed"
}
}
If the client used to make the request also has a store (lrs_id) attached, this will be used to further filter down deletions only to this store.
An intialise request will return a 200 HTTP response with a JSON version of the batch deletion job (see Schema)
TERMINATING BATCH DELETIONS
You may choose to terminate one or all batch deletions for an organisation using the following commands.
Note that if a deletion is currently in progress and a worker job to delete a batch has been issued, up to 1000 records may be deleted before the termination command is respected.
This is due to how we batch the deletions into blocks of 1000. Once a batch has started, it cannot be stopped without manually stopping the Worker’s Node process running the deletion job. However, no subsequent batches will be processed once that batch has finished.
TERMINATING A SINGLE BATCH DELETION
Stop a specific batch deletion from executing any more batches.
TERMINATING ALL BATCH DELETIONS
Stop all batch deletions from executing any more batches
VIEWING BATCH DELETIONS
SCHEMA
Name | Description |
|---|---|
_id | The id of the batch delete job. |
organisation | The id of the organisation that this job belongs to. |
filter | A stringified JSON Mongo query - records matching this filter are deleted |
pageSize | Total records deleted per batch (defaults to 1000, no way to customise outside of code change and rebuild) |
deleteCount | How many records have been deleted so far |
total | Total number of statements found for deletion at initialise |
processing | Boolean value; is there a deletion batch currently being actioned |
done | Boolean value; has the job finished or been terminated |
createdAt | When this document was created |
updatedAt | When this document was last updated |
_The filter field is stored as text to account for . (dot) characters used in query keys - these are invalid in Mongo JSON structures.
EXAMPLE
The Batch Delete model may be retrieved using the GET REST or Connection APIs but other HTTP methods are disabled (PUT, PATCH, DELETE) and are instead replaced by the initialise and terminate routes specified above.
EXAMPLES: (USING THE CONNECTION API)
Note; query parameters should be URL encoded - these examples have not been, for readability
FETCH A PARTICULAR JOB BY _id
FETCH THE 5 MOST RECENTLY COMPLETED/TERMINATED JOBS
FETCH THE 5 MOST RECENTLY CREATED AND UNFINISHED JOBS
DELETION TIME WINDOW
Because deletion can be an intensive job for the database, it is possible to configure Learning Locker to only trigger and process batch deletion during a specified window every day. This can be configured to enable deletions during periods of known low activity (e.g. night time).
To configure this, update the document in the siteSettings collection with the required UTC hour, minute and duration:
The configuration above, for example, would allow deletions to be triggered from midnight to 5am (UTC) everyday. Note that the Scheduler process is required to be run in order outstanding jobs at the start of the window everyday.
Learning Locker and the Squirrel logo are trademark of Learning Pool 2020 | Learning Locker is licensed under GPL 3.0.