iasql vs iac (sync)

blog/2022-07-22-vs-iac.mdx

@@ -0,0 +1,80 @@
+---
+slug: vs-iac
+title: IaSQL vs IaC / Why IaSQL
+authors: [depombo]
+---
+
+Modern, cloud-native IaC tools like Terraform take inspiration from the IaC tools that came before like Chef and Puppet. However, legacy IaC tools were conceived to configure a (somewhat) fixed number of servers in data centers. Then the advent of the cloud came around 2010 and changed everything. Click a button, and get a server. Click a button, and get rid of the server. The commoditization of on-demand, ephemeral servers gave way to a Cambrian explosion of developer tooling at different abstraction (and price) points atop servers in the cloud. However, legacy IaC tooling built for on-premise datacenters didn't anticipate the elasticity of the cloud. Cloud-native IaC tools today are not only used to configure servers in the cloud but many different compute and data services. AWS alone has over 200 services and counting. Each service with a different API.
+
+TODO:
+Puppet snippet VM
+```
+```
+
+TF snippet EC2
+```
+```
+
+One of the biggest assumptions legacy IaC tools make is that the number of servers in data centers doesn't change often. As such, creating a declaration or blueprint of the entire infrastructure works well for on-premise machines. However, applying an infrastructure declaration to a cloud account with many different types of interdependent resources makes it difficult to import and manage the cloud infrastructure that exists outside of said declaration. [IaSQL](https://iasql.com), like IaC tools, allows creating, updating or deleting infrastructure in the cloud. IaSQL does this by using the actual representation of the infrastructure instead of a blueprint. Developers can then model cloud infrastructure in the same way the information in most software systems is represented: as data in a database. This means developers can scale the cloud with a familiar set of developer tools that the Postgres ecosystem has built over many decades, and using one of the most well-known programming languages, SQL.
+
+IaSQL can fully reconcile the current and desired state of infrastructure by providing the familiar `apply` command used in IaC tools like Terraform but also introducing a new input-less `sync` command. We created the IaSQL `sync` command to let different teams create resources outside of IaSQL with any cloud management tool while still inspecting and managing said resources with IaSQL. Infrastructure drift, identifying resources one by one to import them and untamable state files arise when IaC tools try to model infrastructure that exists in the cloud, but not in the IaC declaration. However, these problems go away when infrastructure is simply data in a database.
+
+TODO: current vs desired state cycle diagram
+
+TODO:
+1. create ec2 instance via IaSQL
+2. put an lb in front of it
+3. create an ec2 instance via UI
+4. iasql_sync
+5. put the imported ec2 instance behind the lb and apply
+
+
+### State files vs Databases
+
+Cloud-native IaC tools automatically create state files to reconcile infrastructure that exists in the cloud, but not in the infrastructure declaration. State files in [Terraform](https://www.terraform.io/language/state) and [Pulumi](https://www.pulumi.com/docs/intro/concepts/state/) are JSON files with metadata about cloud accounts that have to be stored separately from the infra declarations, typically in some sort of file storage like S3. The fact that state files are JSON makes them hard to inspect and modify, especially at scale. Terraform has been around for longer than Pulumi and has gone to great lengths to make it palatable to manipulate these machine-generated files using their [CLI state subcommand](https://www.terraform.io/cli/commands/state/mv). This begs the question, why are we using files instead of databases to store and manage large amounts of information about complex systems? Databases were created to replace files when manipulating large amounts of structured information efficiently. Most people that know how to code, regardless of their job title, are familiar with databases in some capacity because databases have been around for decades. The ubiquitousness of databases means an abundance of battle-tested developer tools that are used in most large-scale, stateful software systems and using them enables us to “stand on the shoulders of giants”. IaSQL sets forth that infrastructure is just data and the easiest way to manage any significant amount of data is in a database.
+
+Some organizations can prevent the generation of state files when using IaC tools by making cloud dashboards read-only for all employees and only allowing infrastructure changes to occur through the declaration. However, this means every single team in an organization that can span thousands of people (and software systems) has to adhere to this convention. This can be an untenable premise when workflows span multiple clouds, time zones, deployment strategies, dependencies and versions. Application engineers often need to move quickly and use the cloud UI to spin up infrastructure that could depend on infrastructure previously created through an IaC tool.
+
+![](../static/img/comparison.jpg)
+
+### Import piecemeal vs all at once / Import each resource explicitly or any number of them implicitly
+
+Terraform allows importing existing cloud resources into its state file (not the HCL configuration) one by one with the [`terraform import`](https://www.terraform.io/cli/import/usage) CLI command. 
+
+TODO terraform import example
+```
+```
+
+This piecemeal process makes it very time-consuming to reconcile infrastructure outside the declaration, especially at scale. Pulumi allows [importing](https://www.pulumi.com/docs/guides/adopting/import/) existing resources by identifying each resource via the CLI or calling out each resource via a JSON file or script. Furthermore, importing a resource that is depended on by other pieces of infrastructure often requires a [complex set of steps](https://www.terraform.io/cli/import/usage#complex-imports). On the other hand, IaSQL automatically imports all existing cloud resources as modules representing cloud services are installed without any additional input or steps other than by calling `iasql_sync`. 
+
+TODO pulumi import example
+```
+```
+
+### Infrastructure drift (WIP)
+
+The fact is that someone is going to access live cloud systems at some point to debug or in the case of an emergency. 
+
+IaSQL's automatic import guarantees there is no drift the moment an IaSQL database, which is a Postgres DB with no bells or whistles, is connected to an AWS account.
+
+### Relations and dependencies
+
+IaC tools form DAGs, directed acyclic graphs, under the hood to establish the dependencies between different parts of the infrastructure in a cloud account. IaSQL sets forth using the relational SQL model to encode relationships between pieces of infrastructure using a syntax that is ergonomic and familiar to most developers.
+
+The infrastructure of a production system can get complex quickly. Infrastructure components can be used by different resources. A change to a cloud resource can have undesired side effects, or outages, in the application layer when the relationships across infrastructure components are not clear enough. Infrastructure changes are less prone to errors when the complete representation of all the relevant resources is in a relational database. Furthermore, debugging for outliers or misconfiguration can be easily done with SQL. Sometimes the issue under the surface is not misconfiguration, but just configuration drift which can happen when a cloud resource has not been updated in a long time or since its creation.
+
+### Type Safety (WIP)
+
+The advent of typed PL has shown the importance of being able to catch a subset of all errors compile-time. IaC tools are akin to scripting languages here since any errors generated in their internal representations DAGs are only surfaced at runtime.
+
+### Backups and disaster recovery (WIP)
+
+Every production system should have contingency plans in case issues arise. Cloud outages and human errors can delete or corrupt infrastructure. IaSQL provides a recovery system plan based on up-to-date database backups. It is trivial to apply a snapshot after an outage and re-deploy the entire production infrastructure in a simple step.
+
+### What now?
+
+In IaSQL, cloud infrastructure is data through a 2-way binding between a cloud account and a PostgreSQL database. The ability to represent cloud infrastructure in a PostgreSQL database through a 2-way connection allows making CRUD changes while also tracking changes done elsewhere. Cloud infrastructure declarations are great in theory, but they don't describe what is *actually* in the cloud account without state files, infrastructure drift or time-consuming import migrations. SQL is the lingua franca of data which means that many developers are already familiar with IaSQL. Furthermore, SQL offers a familiar, type-safe and ergonomic way to encode dependencies across infrastructure parts in a way that configuration-based syntaxes cannot.
+
+We hope that with IaSQL, developers and DevOps teams alike will bring about a new era for the cloud. Building on the cloud will be more productive, more enjoyable, and more familiar. Modern cloud software will shift from being a black art that only a select few can modify and master, to something just another elegant software system that any developer can manage.
+
+The proposition is ambitious. IaSQL still has a long way to go. We only support parts of AWS and we are not even at a 0.1 version yet! However, we look forward to a time when we can treat infrastructure like just another database in the backend managed with pure SQL or whatever programming language fits my stack using the same old database management tools.