This is about the same order of magnitude as the number of VMs supported in a
host

Signed-off-by: Pau Ruiz Safont <[email protected]>

This helps when running it to observe behaviour of the library

Signed-off-by: Pau Ruiz Safont <[email protected]>

Add standard http attributes to from opentelemetry such as:

- `http.request.method`
- `http.request.header.content-type`;
- `http.request.body.size`;
- `http.request.header.user-agent`;
- `http.request.header.(key)`.

This improves debuggability associated with client requests.

Signed-off-by: Gabriel Buica <[email protected]>

This reuses the re-key process to gather current contents of the cache and
depending on the amount of elements pending it
1. Bypasses the cache completely if it's empty
2. Uses the queue if all the elements fit in the queue
3. Starts as before if there are more elements

Signed-off-by: Pau Ruiz Safont <[email protected]>

Unknown directories detected in the cache will get logged, and at the
appropriate times deleted, along with all the files contained within.

Signed-off-by: Pau Ruiz Safont <[email protected]>

Previously a logline was created per push attempt, now up to two per failure
period are done: one on the first failure, and one on recovery.

Signed-off-by: Pau Ruiz Safont <[email protected]>

Now temporary files are an internal detail of the persistence function. The
rest of code needs not be aware of it: if some of temp files linger in the
filesystem means that the process did not acknowledge a requested write, so the
domain could not have possibly acted on the write. Delete these as this does
not incur in any data corruption in the domain side.

Signed-off-by: Pau Ruiz Safont <[email protected]>

With *hybrid source format in export, the following cases are supported:
1. nbdhybrid: QCOW2 -> NBD device in dom0 -> exported file
2. nbdhybrid: VHD -> NBD device in dom0 -> exported file
3. hybrid: VHD -> blktap device in dom0 -> exported file

The case 2 above can't support an optional parameter "base".
This parameter holds the ID of another VHD VDI. When it is passed, the
export will only write the differences between "source" and "base" to
the destination file.

As a short-term solution, in case 2 above, when the "base" is passed,
the source format is changed to "hybrid" in this commit.
This can work because:
1. the comparsion on blocks required by "base" is supported by "hybrid";
2. the raw data from NBD device and blktap (Frankentap) device are same;
3. the sparseness information of the source required in case 2 can be
   get by either NBD interface (nbdhybrid) or VHD parsing (hybrid).

Signed-off-by: Ming Lu <[email protected]>

waitpid_nohang closes the socket by calling waitpid and then calls
clear_nonblock on that socket, which fails.

waitpid_nohang's policy around closing the socket is not obvious enough.
Make it more explicit by not calling waitpid but inlining code and
looking at the different cases. The behaviour is still to close the
socket when the child process has terminated and not otherwise.

Logging can be re-enabled if we have to come back to this.

Signed-off-by: Christian Lindig <[email protected]>

…384483

CA-384483: Can't export VDI to VHD file with base VDI

Signed-off-by: Lunfan Zhang <[email protected]>

Signed-off-by: Benjamin Reis <[email protected]>

Slightly different from missing file, add missing path.

Signed-off-by: Frediano Ziglio <[email protected]>

Clean even if user press Ctrl-C, not only sending TERM signal.
Remove possible stale unix socket to allow to run multiple times.

Signed-off-by: Frediano Ziglio <[email protected]>

Make sure we get the correct output from program launched.

Signed-off-by: Frediano Ziglio <[email protected]>

Make sure program receives the string we are sending.

Signed-off-by: Frediano Ziglio <[email protected]>

Requires libxml2-devel installed.

Signed-off-by: Edwin Török <[email protected]>
(cherry picked from commit 0008395)

The C SDK build was failing with a recent GCC on Fedora39 like this:
```
src/xen_common.c: In function ‘xen_session_logout’:
src/xen_common.c:298:5: error: ‘xen_call_’ accessing 16 bytes in a region of size 0 [-Werror=stringop-overflow=]
  298 |     xen_call_(session, "session.logout", params, 0, NULL, NULL);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

Use `NULL` instead of a VLA of size 0.

Signed-off-by: Edwin Török <[email protected]>
(cherry picked from commit bfe78bd)

…dora-fix

Port of "Fix C SDK build on Fedora39" to `master`

Domains using TPMs need to have SWTPM always available. Encode this information
in the service file so the services are shut down at the correct time while
powering off the hosts.

Signed-off-by: Pau Ruiz Safont <[email protected]>

I am not sure where this restriction came from. It was introduced in the
same commit that introduced the metadata-export feature itself, without
explanation. Full exports of snapshots are already allowed.

Signed-off-by: Rob Hoes <[email protected]>

This allows clients to know which snapshots they knowingly just created and act
on them instead of looking again into the db and try to divine which snapshots
the call created.

Signed-off-by: Pau Ruiz Safont <[email protected]>

This allows clients to detect that the newer device type exclusions for export
are available.

Signed-off-by: Rob Hoes <[email protected]>

This allows to orchestrators to copy devices like VBDs from the VM, export the
metadata without the disk, import the VM to any pool, and finally add the
saved VBDs to the new VM before booting it.

Signed-off-by: Pau Ruiz Safont <[email protected]>

This allows to easily test the http endpoint. Also advertises the metadata
parameter on these calls

Signed-off-by: Pau Ruiz Safont <[email protected]>

Use the helper functions to process parameters, use atomics instead of ad-hoc
references for counting in possibly-parallel case

Signed-off-by: Pau Ruiz Safont <[email protected]>

Signed-off-by: Pau Ruiz Safont <[email protected]>

Signed-off-by: Pau Ruiz Safont <[email protected]>

Signed-off-by: Pau Ruiz Safont <[email protected]>

Tooling complains about using 1970 as the date otherwise

Signed-off-by: Pau Ruiz Safont <[email protected]>

Signed-off-by: Konstantina Chremmou <[email protected]>

…ernal headers from the public ones).

Signed-off-by: Konstantina Chremmou <[email protected]>

Signed-off-by: Konstantina Chremmou <[email protected]>

Protocol_unix spawns a new thread for each request and already processes requests concurrently.
However Protocol_{lwt,async} was waiting for each request to be fully processed before fetching and dispatching the next one.

Change `iter` to `iter_donwait`, run all promises in parallel, and don't wait for any of them to complete.
Exceptions have to go somewhere though, so set up an Lwt.async_exception_hook that logs the exception.

Note: this might need some throttling so that sending lots of RPC calls doesn't cause the server to run OOM.But Protocol_unix already has this problem.

Signed-off-by: Edwin Török <[email protected]>

Previously we had problems with making message switch process multiple
messages at the same time, especially with the xapi-storage-script. It
would still be good to have this feature, but be more conservative and
make it off by default.

Signed-off-by: Vincent Liu <[email protected]>

Signed-off-by: Vincent Liu <[email protected]>

Signed-off-by: Danilo Del Busso <[email protected]>

Signed-off-by: Danilo Del Busso <[email protected]>

…witch-optional

CP-47033: Make message switch concurrent processing optional

OTEL_RESOURCE_ATTRIBUTES is in W3CBaggageFormat and thus uses %.
ConfigParser attempts to interpolate these % signs but we want them to
be left alone, so set interpolation=None

Signed-off-by: Steven Woods <[email protected]>

…-391485

CA-391485: Avoid InterpolationSyntaxError by turning off interpolation

A recent change added the dependency

Signed-off-by: Pau Ruiz Safont <[email protected]>

There's no need for suffix, Ocaml has different namespace for types.
This was suggested as part of a code review.

Signed-off-by: Frediano Ziglio <[email protected]>

…wer!

Don't use string comparisons for versions, "17" < "2", but 17 > 2!

Signed-off-by: Edwin Török <[email protected]>

…enver

xenopsd: fix Xen version comparison. 4.17 is > 4.2, not lower!

The lock implementation for `protocol_async` was not correct, hence
adding the test suggested by Edwin to trigger race.

Test works by trying to acquire a standard Ocaml mutex in a critical
section provided by the `with_lock` implementation. This means that any
failed attempt to acquire the lock implies the `with_lock`
implementation does not provide mutual exclusion.

Signed-off-by: Vincent Liu <[email protected]>

…CP-47660

CP-47660 define anti-affinity feature

Not only print the time taken for program execution but also
check the process was stopped not too early and not too late.

Signed-off-by: Frediano Ziglio <[email protected]>

Signed-off-by: Mark Syms <[email protected]>

Add standard http attributes to from opentelemetry such as:

- `network.local.address`;
- `network.local.port`;
- `network.peer.address`;
- `network.peer.port`.

 This improves debuggability associated with client requests.

Signed-off-by: Gabriel Buica <[email protected]>

…/CP-46576

This is useful for unit-testing, and more.
Printing helps with displaying the values in unit-tests, and the all helps to
get exhaustive checks without having to manually sync the types

Signed-off-by: Pau Ruiz Safont <[email protected]>

Signed-off-by: Pau Ruiz Safont <[email protected]>

The list in xapi_vm_lifecycle now contains ignored operations, this allows
users to easily see which operations are excluded from the database, decide
whether they're worth exposing them and change the code accordingly to
accomodate for them.

Signed-off-by: Pau Ruiz Safont <[email protected]>

…sync-race

Signed-off-by: Konstantina Chremmou <[email protected]>

Signed-off-by: Pau Ruiz Safont <[email protected]>

test: add tests for allowed VM operations

…dk-c

CA-387885 and templatization of the C SDK

Attempts to split the tracing library into components and exporter
parts.

While trying to instrument `forkhelpers.ml` with tracing, I found a
cycle dependency: `Tracing(Exporter)` -> `Open_uri` -> `Stunnel` ->
`Forkhelpers`. This splits the tracing library between exporting
functionality and components.

Signed-off-by: Gabriel Buica <[email protected]>

Small improvements to `tracing_export.ml` and `tracing_exoprt.mli`.

Issues found while splitting the library such as: missing documentation
and small code refactoring for better readability.

Signed-off-by: Gabriel Buica <[email protected]>

Adds `with_tracing` helper function to `forkhelpers.ml`.

This is to show that there are no more cycle dependecies between
`tracing` library and `forexecd` and it will be used in a follow-up PR
to instrument `forkhelpers` with tracing.

Signed-off-by: Gabriel Buica <[email protected]>

As discussed here: xapi-project/xapi-project.github.io#286

New methods:
- `PCI.disable_dom0_access`: Hide a PCI from dom0 kernel
- `PCI.enable_dom0_access`: Unhide a PCI from dom0 kernel
- `PCI.get_dom0_access_status`: Return a PCI device dom0 access status
The answer is no longer based on a DB field but by comparing the `/proc/cmdline`
and the xen cmdline.

Deprecated methods:
- `PGPU.disable_dom0_acces`
- `PGPU.enable_dom0_access`
The methods should be called on the PCI belonging to the PGPU instead

Deprecated field:
- `PGPU.dom0_access`

The deprecated methods still work since they now call the PCI methods.
The PGPU field is kept up to date with the statuss returned by the PCI methods.

Corresponding xe CLI calls have been implemented.

Signed-off-by: Benjamin Reis <[email protected]>

It was allocating a string for each character, only to then immediately add it to a buffer.
Instead add the character to the buffer directly.

The temporary string was probably created to factor out 'Buffer.add',
but I'd rather copy Buffer.add 4x times than have the perf hit.

Further optimizations might be possible here as the comments says,
but this is a low-risk, obvious optimization.

Signed-off-by: Edwin Török <[email protected]>

Optimistically check for the presence of escape characters before escaping.
This avoids rebuilding the string when escapable characters are not present (the common case).

Signed-off-by: Edwin Török <[email protected]>

Check for the presence of the escape character before rebuilding the string

Signed-off-by: Edwin Török <[email protected]>

…expr

IH-553: Optimize Sexpr.{escape,unescape}

…/CP-48195-v2

CP-48195: Split `tracing` library to avoid future cyclic dependencies

Fix shellcheck warnings and bugs.

Signed-off-by: Edwin Török <[email protected]>