How gcsfuse identifies if a GCS object is file or directory?

[raj-prince]

I would like to understand - how gcsfuse detects if a GCS object is file or directory.

[raj-prince]

Let's first understand the important terminologies:

(a) Explicit Directory - GCSFuse treats an object with trailing delimiter (/) as explicit directory. If you create a directory via GCSFuse it creates an object in GCS with trailing delimiter (/). E.g. - a/b/c/

(b) File - GCSFuse treats an object without trailing delimiter (/) as a file. E.g. - a/b/c/d.txt

(c) Implicit Directory - GCSFuse infers the existence of implicit directory if there are objects within a "path" that share a common prefix. E.g. If a bucket contains only one object a/b/c/d.txt, GCSFuse treats a/, a/b, and a/b/c/ as implicit directories.

GCSFuse only understand the concept of implicit directory when bucket is mounted with--implicit-dirs flag.

CASE 1: if bucket is mounted without --implicit-dirs then two stat calls are required to know if the object name is file or directory one with / and other without /.

E.g. To know if a/b/c is file or directory, we need to make two GetObjectMetadata calls - (i) without / GetObjectMetadata(a/b/c) and (ii) with / GetObjectMetadata(a/b/c/).

• if only (i) succeeds that means file,
• if only (ii) succeeds that means directory and
• if both succeeds we consider as a directory but later caught as conflicting names while listing as it's not the right scenario.

CASE 2: if bucket is mounted with --implicit-dirs then one stat call and one list call is required to know if the object name is file or directory - one with / and other without /.

E.g. To know if a/b/c is file or directory, we need to make two calls (i) GetObjectMetadata(a/b/c) without '/' and (ii) List(a/b/c/) with '/'.

• if only (i) succeeds that means file,
• if only (ii) succeeds that means directory and
• if both succeeds, we consider as a directory but later caught as conflicting names while listing as it's not the right scenario.

Summary:
GCSFuse identifies if a GCS object is a file or directory by making two GCS calls - (i) 1 stat call to check if the object is file + (ii) 1 stat or 1 list call to check if the object is directory.