You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So I've done a bunch of design work around the architecture / strategy and ended up with a nice solution for developing the backend going forwards which I'll get into below.
First of all, the main driving motivations behind this are to:
Decouple the database models from the API models
This causes a bunch of issues because you have to change both at once, you can't have API only fields without also putting them in the database model, etc.
Remove hard dependencies on the database for using the API models on their own and hence allow API library developers to consume these models directly from their libraries
By splitting API models into their own crate, developers can freely pull this in without pulling in hundreds of other unrelated dependencies which they don't need. As well as this, developers need not manually figure out their models anymore.
Allow versioning and improving the API independent of database models.
This will give us the freedom to improve the API without needing database migrations and as such, we could deploy multiple versions of the API with different models as people transition over in the future.
Introduce testing on top of the core database models and then eventually on top of the API as well.
There are a couple conditions with this refactor though:
It must be ongoing not a one-shot; this is a lot of work that can't be done in one go.
As per the previous point, everything must still work as things are refactored.
To reiterate, do not break the API! But instead complement new tests and fix anything wrong.
So, here's what's changed:
The project structure is now as follows:
|- crates
|- core
|- database: Database Implementation (In-Memory and MongoDB)
|- result: Result and Error types
|- models: API Models
|- presence: Global user presence system
|- quark: (now deprecated) the old "core" crate for everything
|- delta: API, nothing changes here
|- bonfire: events, nothing changes here
API routes now have access to rocket::State<revolt_database::Database> for any new database model operations.
There is a new in-memory database for development purposes as opposed to the old useless "dummy db".
All (new and current that have been ported) database models are and must be unit tested! 🎉
Continous integration has already been setup to run unit tests against the in-memory database and MongoDB.
Quark errors may be mapped to from the new revolt_result::Error type, do it as such:
// db comes from revolt_database// we use Error::from_core with map_err to convert
db.fetch_bot(&target.id).await.map_err(Error::from_core)?;
Since the database and API models are separate, you can convert between them by using .into() or the appropriate function if more than one object is required from the database!
There are also some things that need to be sorted out still:
Relicense revolt-result crate to be MIT
Relicense revolt-models crate to "be MIT" (without from_database feature) or "MIT / AGPL" (with from_database feature)
I'd need to read up a bit on this and check if I need to do anything special here with how Cargo dependencies work, in theory I should just need to add a notice that while revolt-models itself is MIT, it has an optional dependency to an AGPL licensed crate.
You can read the new code on the refactor/split-project-into-core-crates branch (which will be merged in the near future).
These changes will also be used to test #210 (webhooks) before it is merged in.
Future plans:
Clean up the bad things about the API, for example, getting rid of _id in favour of id or introducing a better system for remvoing fields such as sending empty strings. As mentioned earlier, we could provide a "v1" and "v2" API endpoint to avoid disruption since database models would be separate.
insertish
changed the title
Refactoring the project into smaller distinct crates for developer productivity.
🎊 Refactoring the project into smaller distinct crates for developer productivity.
Apr 22, 2023
Just added a presence crate to core with a unit test covering the whole crate.
Found one issue with how I was handling flags in the mean time and that's been fixed now.
So I've done a bunch of design work around the architecture / strategy and ended up with a nice solution for developing the backend going forwards which I'll get into below.
First of all, the main driving motivations behind this are to:
This causes a bunch of issues because you have to change both at once, you can't have API only fields without also putting them in the database model, etc.
By splitting API models into their own crate, developers can freely pull this in without pulling in hundreds of other unrelated dependencies which they don't need. As well as this, developers need not manually figure out their models anymore.
This will give us the freedom to improve the API without needing database migrations and as such, we could deploy multiple versions of the API with different models as people transition over in the future.
There are a couple conditions with this refactor though:
So, here's what's changed:
rocket::State<revolt_database::Database>
for any new database model operations.Continous integration has already been setup to run unit tests against the in-memory database and MongoDB.
revolt_result::Error
type, do it as such:.into()
or the appropriate function if more than one object is required from the database!There are also some things that need to be sorted out still:
revolt-result
crate to be MITrevolt-models
crate to "be MIT" (withoutfrom_database
feature) or "MIT / AGPL" (withfrom_database
feature)I'd need to read up a bit on this and check if I need to do anything special here with how Cargo dependencies work, in theory I should just need to add a notice that while
revolt-models
itself is MIT, it has an optional dependency to an AGPL licensed crate.You can read the new code on the refactor/split-project-into-core-crates branch (which will be merged in the near future).
These changes will also be used to test #210 (webhooks) before it is merged in.
Future plans:
_id
in favour ofid
or introducing a better system for remvoing fields such as sending empty strings. As mentioned earlier, we could provide a "v1" and "v2" API endpoint to avoid disruption since database models would be separate.Note: If I forgot something, I'll edit the post and also mention it in the comments that it's changed.
Remaining models and implementations:
Other issues to tackle:
revolt-result
crate to MIT #277revolt-models
crate (excl. from_database implementation) to MIT #278The text was updated successfully, but these errors were encountered: