Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pulumi: Migrate pyproject to PyPA standard and hatch #1506

Merged
merged 86 commits into from
Jul 27, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
00aa33c
Migrate pyproject to PyPA standard and hatch
JimMadge Jul 17, 2023
8fee901
Remove flake8 configuration dotfile
JimMadge Jul 20, 2023
8055ddb
Correct lint env dependencies
JimMadge Jul 20, 2023
b59db91
Set max line length to 120
JimMadge Jul 20, 2023
cde7ec7
Use hatch for python package linting
JimMadge Jul 20, 2023
0eadf92
Remove ignore for B027
JimMadge Jul 24, 2023
abd6b5f
Add comments for Ruff rules
JimMadge Jul 24, 2023
818c01a
Improve version specification
JimMadge Jul 24, 2023
ade93ab
Remove ignore for FBT003
JimMadge Jul 24, 2023
ed13799
Fix version specifier
JimMadge Jul 24, 2023
c543914
Run black and ruff
JimMadge Jul 24, 2023
d802fc5
Remove unfixable for F401
JimMadge Jul 24, 2023
033cb7a
Remove unused modules from mypy overrides
JimMadge Jul 24, 2023
362e3a7
Fix ruff errors in types.py
JimMadge Jul 24, 2023
f3004f0
Fix Ruff errors in logger.py
JimMadge Jul 24, 2023
9b9e9df
Add missing version.py
JimMadge Jul 24, 2023
61dc3de
Fix Ruff errors in pulumi_stack.py
JimMadge Jul 24, 2023
6ba4bad
Fix Ruff errors in ssl_certificate.py
JimMadge Jul 24, 2023
f7e04b3
Fix Ruff errors in remote_powershell.py
JimMadge Jul 24, 2023
601b5c0
Fix ruff errors in file_share_file.py
JimMadge Jul 24, 2023
33cb8ce
Fix Ruff errors in dsh_resource_provider.py
JimMadge Jul 24, 2023
df9bde1
Fix Ruff errors in compiled_dsc.py
JimMadge Jul 24, 2023
150ad4d
Fix Ruff errors in blob_container_acl.py
JimMadge Jul 24, 2023
4a84df8
Fix Ruff errors in azuread_applications.py
JimMadge Jul 24, 2023
e290420
Fix Ruff errors in virtual_machine.py
JimMadge Jul 24, 2023
7e9087a
Fix Ruff errors in sre_software_repositories.py
JimMadge Jul 24, 2023
6e83d58
Fix Ruff errors in sre_networking.py
JimMadge Jul 24, 2023
9aef2ec
Fix Ruff errors in sre_remote_desktop.py
JimMadge Jul 24, 2023
53f197e
Fix Ruff errors in sre_monitoring.py
JimMadge Jul 24, 2023
4dffc2b
Fix ARG002 errors
JimMadge Jul 24, 2023
bfecf96
Fix Ruff errors in sre_hedgedoc_server.py
JimMadge Jul 24, 2023
d713b9b
Fix Ruff errors in sre_gitea_server.py
JimMadge Jul 24, 2023
d5071d9
Fix Ruff errors in sre_data.py
JimMadge Jul 24, 2023
5d4df58
Fix Ruff errors in sre_application_gateway.py
JimMadge Jul 24, 2023
2e82f65
Fix line length errors
JimMadge Jul 24, 2023
34b09ef
Fix Ruff errors in validators.py
JimMadge Jul 24, 2023
c5bb51d
Fix Ruff errors in miscellaneous.py
JimMadge Jul 24, 2023
a62d52e
Fix Ruff errors in azure_postgresql_database.py
JimMadge Jul 24, 2023
a72b446
Fix Ruff errors in azure_ipv4_range.py
JimMadge Jul 24, 2023
b503811
Fix Ruff errors in graph_api.py
JimMadge Jul 24, 2023
120fe72
Fix Ruff errors in azure_cli.py
JimMadge Jul 24, 2023
40e2329
Use PEP naming convention for Exceptions
JimMadge Jul 24, 2023
3a96088
Fix Ruff errors in config.py
JimMadge Jul 24, 2023
18ab14d
Ignore error for print statement
JimMadge Jul 24, 2023
1980c74
Ignore A003 where it improves clarity
JimMadge Jul 24, 2023
fea84c1
Ignore magic number where it is not resused
JimMadge Jul 24, 2023
6eccea6
Run fmt script
JimMadge Jul 24, 2023
19bad5a
Fix whitespace in string
JimMadge Jul 24, 2023
3bc62f8
Improve subprocess output handling
JimMadge Jul 25, 2023
ef21e9a
Remove unnecessary f-string conversions
JimMadge Jul 25, 2023
044d82b
Remove __about__.py
JimMadge Jul 25, 2023
741290b
Allow relative imports of siblings
JimMadge Jul 25, 2023
90b87da
Remove remaining standard library comment
JimMadge Jul 25, 2023
1d36ab4
Add docstring for truncate_tokens
JimMadge Jul 25, 2023
b9ea215
Enable black string normalisation
JimMadge Jul 25, 2023
0a5d27c
Update linting workflow to run hatch style script
JimMadge Jul 25, 2023
f6ff62e
Move inline comment
JimMadge Jul 25, 2023
40ae88b
Use typing.Optional for Typer annotations
JimMadge Jul 25, 2023
4058ee0
Explain that check_returncode may raise exception
JimMadge Jul 25, 2023
e856efa
Tidy LDAP search filter
JimMadge Jul 25, 2023
362eafd
Clarify comment
JimMadge Jul 25, 2023
6a73875
Use combine-as-imports isort option
JimMadge Jul 25, 2023
65280d4
Restore required arguments for Pulumi 'callbacks'
JimMadge Jul 25, 2023
0066d7b
Add missing id_ argument
JimMadge Jul 25, 2023
6af37b0
Remove unused YamlType
JimMadge Jul 25, 2023
62a5d61
Run linting on python-migration branch
JimMadge Jul 25, 2023
18c7732
Fix Typer annotations
JimMadge Jul 26, 2023
ddb3fa6
Improve security rule description
JimMadge Jul 26, 2023
564e536
Correct import path
JimMadge Jul 26, 2023
deaf911
Use next() to lazily fetch first list element
JimMadge Jul 26, 2023
03cc8a1
Use default black line length, ignore in ruff
JimMadge Jul 26, 2023
a268b95
:rotating_light: Increase spacing before comment
jemrobinson Jul 26, 2023
1541c28
:bug: Fix use of subprocess ENV
jemrobinson Jul 26, 2023
386b680
:bug: Fix as_dict function which was broken by variable name change
jemrobinson Jul 26, 2023
5edec7f
:bug: Fix missing timezone import
jemrobinson Jul 26, 2023
95c96a4
:bug: Remove incorrect SHM name string parameter from SHM components
jemrobinson Jul 26, 2023
804f43f
:bug: SHMUpdateServersComponent should take stack_name not shm_name a…
jemrobinson Jul 26, 2023
8b75f9b
:truck: Better variable name
jemrobinson Jul 26, 2023
b500d37
:bug: Separate SHM and SRE stack YAML files
jemrobinson Jul 26, 2023
dc209a6
:rotating_light: Remove empty line
jemrobinson Jul 26, 2023
c2a494a
:bug: Remove incorrect SHM/SRE name string parameter from SRE components
jemrobinson Jul 26, 2023
b6d6261
:rotating_light: Fix README
jemrobinson Jul 26, 2023
386f37d
:bug: Removed incorrect parameters from SREGiteaServerComponent and S…
jemrobinson Jul 26, 2023
b75503f
:recycle: Move sre_name from component constructor to props for SRERe…
jemrobinson Jul 26, 2023
31f05cf
:recycle: Move sre_name from component constructor to props for SRENe…
jemrobinson Jul 26, 2023
6bb9fd0
:see_no_evil: Add ruff cache to gitignore
jemrobinson Jul 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions .flake8

This file was deleted.

12 changes: 7 additions & 5 deletions .github/workflows/lint_code.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,14 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Install requirements
shell: bash
run: pip install flake8
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.11
- name: Install hatch
run: pip install hatch
- name: Lint Python
shell: bash
run: flake8 . --statistics --count
run: hatch run lint:all

lint_shell:
runs-on: ubuntu-latest
Expand Down
2 changes: 2 additions & 0 deletions data_safe_haven/__about__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
__version_info__ = ("5", "0", "0")
__version__ = ".".join(__version_info__)
5 changes: 3 additions & 2 deletions data_safe_haven/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Data Safe Haven"""
import pkg_resources

__version__ = pkg_resources.get_distribution("data-safe-haven").version
from data_safe_haven.version import __version__, __version_info__

__all__ = ["__version__", "__version_info__"]
2 changes: 1 addition & 1 deletion data_safe_haven/administration/users/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
from .user_handler import UserHandler
from data_safe_haven.administration.users.user_handler import UserHandler
JimMadge marked this conversation as resolved.
Show resolved Hide resolved

__all__ = ["UserHandler"]
36 changes: 12 additions & 24 deletions data_safe_haven/administration/users/active_directory_users.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
"""Interact with users in an Azure Active Directory"""
# Standard library imports
import pathlib
from typing import Any, Optional, Sequence
from collections.abc import Sequence
from typing import Any

from data_safe_haven.administration.users.research_user import ResearchUser
JimMadge marked this conversation as resolved.
Show resolved Hide resolved

# Local imports
from data_safe_haven.config import Config
from data_safe_haven.external import AzureApi
from data_safe_haven.functions import b64encode
from data_safe_haven.pulumi import PulumiSHMStack
from data_safe_haven.utility import FileReader, Logger
from .research_user import ResearchUser


class ActiveDirectoryUsers:
Expand All @@ -25,19 +27,13 @@ def __init__(
shm_stack = PulumiSHMStack(config)
self.azure_api = AzureApi(config.subscription_name)
self.logger = Logger()
self.resource_group_name = shm_stack.output("domain_controllers")[
"resource_group_name"
]
self.resources_path = (
pathlib.Path(__file__).parent.parent.parent / "resources"
).resolve()
self.resource_group_name = shm_stack.output("domain_controllers")["resource_group_name"]
self.resources_path = (pathlib.Path(__file__).parent.parent.parent / "resources").resolve()
self.vm_name = shm_stack.output("domain_controllers")["vm_name"]

def add(self, new_users: Sequence[ResearchUser]) -> None:
"""Add list of users to local Active Directory"""
add_users_script = FileReader(
self.resources_path / "active_directory" / "add_users.ps1"
)
add_users_script = FileReader(self.resources_path / "active_directory" / "add_users.ps1")
csv_contents = ["SamAccountName;GivenName;Surname;Mobile;Email;Country"]
for user in new_users:
if (
Expand Down Expand Up @@ -70,11 +66,9 @@ def add(self, new_users: Sequence[ResearchUser]) -> None:
for line in output.split("\n"):
self.logger.parse(line)

def list(self, sre_name: Optional[str] = None) -> Sequence[ResearchUser]:
def list(self, sre_name: str | None = None) -> Sequence[ResearchUser]:
JimMadge marked this conversation as resolved.
Show resolved Hide resolved
"""List users in a local Active Directory"""
list_users_script = FileReader(
self.resources_path / "active_directory" / "list_users.ps1"
)
list_users_script = FileReader(self.resources_path / "active_directory" / "list_users.ps1")
script_params = {"SREName": sre_name} if sre_name else {}
output = self.azure_api.run_remote_script(
self.resource_group_name,
Expand All @@ -100,9 +94,7 @@ def list(self, sre_name: Optional[str] = None) -> Sequence[ResearchUser]:

def register(self, sre_name: str, usernames: Sequence[str]) -> None:
"""Add usernames to SRE security group"""
register_users_script = FileReader(
self.resources_path / "active_directory" / "add_users_to_group.ps1"
)
register_users_script = FileReader(self.resources_path / "active_directory" / "add_users_to_group.ps1")
output = self.azure_api.run_remote_script(
self.resource_group_name,
register_users_script.file_contents(),
Expand All @@ -114,9 +106,7 @@ def register(self, sre_name: str, usernames: Sequence[str]) -> None:

def remove(self, users: Sequence[ResearchUser]) -> None:
"""Remove list of users from local Active Directory"""
remove_users_script = FileReader(
self.resources_path / "active_directory" / "remove_users.ps1"
)
remove_users_script = FileReader(self.resources_path / "active_directory" / "remove_users.ps1")
usernames_b64 = b64encode("\n".join(user.username for user in users))
output = self.azure_api.run_remote_script(
self.resource_group_name,
Expand All @@ -136,9 +126,7 @@ def set(self, users: Sequence[ResearchUser]) -> None:

def unregister(self, sre_name: str, usernames: Sequence[str]) -> None:
"""Remove usernames from SRE security group"""
register_users_script = FileReader(
self.resources_path / "active_directory" / "remove_users_from_group.ps1"
)
register_users_script = FileReader(self.resources_path / "active_directory" / "remove_users_from_group.ps1")
output = self.azure_api.run_remote_script(
self.resource_group_name,
register_users_script.file_contents(),
Expand Down
31 changes: 9 additions & 22 deletions data_safe_haven/administration/users/azure_ad_users.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
"""Interact with users in an Azure Active Directory"""
# Standard library imports
from typing import Any, Sequence
from collections.abc import Sequence
from typing import Any

from data_safe_haven.administration.users.research_user import ResearchUser

# Local imports
from data_safe_haven.external import GraphApi
from data_safe_haven.functions import password
from data_safe_haven.utility import Logger
from .research_user import ResearchUser


class AzureADUsers:
Expand All @@ -25,11 +27,7 @@ def __init__(
def add(self, new_users: Sequence[ResearchUser]) -> None:
"""Add list of users to AzureAD"""
# Get the default domain
default_domain = [
domain["id"]
for domain in self.graph_api.read_domains()
if domain["isDefault"]
][0]
default_domain = [domain["id"] for domain in self.graph_api.read_domains() if domain["isDefault"]][0]
for user in new_users:
request_json = {
"accountEnabled": user.account_enabled,
Expand All @@ -41,12 +39,8 @@ def add(self, new_users: Sequence[ResearchUser]) -> None:
"userPrincipalName": f"{user.username}@{default_domain}",
}
if user.email_address and user.phone_number:
self.graph_api.create_user(
request_json, user.email_address, user.phone_number
)
self.logger.info(
f"Ensured user '{user.preferred_username}' exists in AzureAD"
)
self.graph_api.create_user(request_json, user.email_address, user.phone_number)
self.logger.info(f"Ensured user '{user.preferred_username}' exists in AzureAD")
# Decorate all users with the Linux schema
self.set_user_attributes()
# # Ensure that all users belong to an associated group the same name and UID
Expand All @@ -63,11 +57,7 @@ def list(self) -> Sequence[ResearchUser]:
account_enabled=user_details["accountEnabled"],
email_address=user_details["mail"],
given_name=user_details["givenName"],
phone_number=(
user_details["businessPhones"][0]
if len(user_details["businessPhones"])
else None
),
phone_number=(user_details["businessPhones"][0] if len(user_details["businessPhones"]) else None),
JimMadge marked this conversation as resolved.
Show resolved Hide resolved
sam_account_name=(
user_details["onPremisesSamAccountName"]
if user_details["onPremisesSamAccountName"]
Expand All @@ -77,10 +67,7 @@ def list(self) -> Sequence[ResearchUser]:
user_principal_name=user_details["userPrincipalName"],
)
for user_details in user_list
if (
user_details["onPremisesSamAccountName"]
or user_details["isGlobalAdmin"]
)
if (user_details["onPremisesSamAccountName"] or user_details["isGlobalAdmin"])
]

def remove(self, users: Sequence[ResearchUser]) -> None:
Expand Down
13 changes: 6 additions & 7 deletions data_safe_haven/administration/users/guacamole_users.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Standard library imports
import pathlib
from typing import Any, Optional, Sequence
from collections.abc import Sequence
from typing import Any

from data_safe_haven.administration.users.research_user import ResearchUser

# Local imports
from data_safe_haven.config import Config
from data_safe_haven.external import AzurePostgreSQLDatabase
from data_safe_haven.pulumi import PulumiSREStack
from .research_user import ResearchUser


class GuacamoleUsers:
Expand All @@ -20,12 +22,9 @@ def __init__(self, config: Config, sre_name: str, *args: Any, **kwargs: Any):
sre_stack.output("remote_desktop")["resource_group_name"],
config.subscription_name,
)
self.users_: Optional[Sequence[ResearchUser]] = None
self.users_: Sequence[ResearchUser] | None = None
self.postgres_script_path: pathlib.Path = (
pathlib.Path(__file__).parent.parent.parent
/ "resources"
/ "remote_desktop"
/ "postgresql"
pathlib.Path(__file__).parent.parent.parent / "resources" / "remote_desktop" / "postgresql"
)
self.sre_name = sre_name
self.group_name = f"Data Safe Haven Users SRE {sre_name}"
Expand Down
18 changes: 9 additions & 9 deletions data_safe_haven/administration/users/research_user.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# Standard library imports
from typing import Any, Optional
from typing import Any


class ResearchUser:
def __init__(
self,
account_enabled: Optional[bool] = None,
country: Optional[str] = None,
email_address: Optional[str] = None,
given_name: Optional[str] = None,
phone_number: Optional[str] = None,
sam_account_name: Optional[str] = None,
surname: Optional[str] = None,
user_principal_name: Optional[str] = None,
account_enabled: bool | None = None,
country: str | None = None,
email_address: str | None = None,
given_name: str | None = None,
phone_number: str | None = None,
sam_account_name: str | None = None,
surname: str | None = None,
user_principal_name: str | None = None,
) -> None:
self.account_enabled = account_enabled
self.country = country
Expand Down
Loading