-
Notifications
You must be signed in to change notification settings - Fork 619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error when implementing probabilistic augmentations #5544
Comments
Hi @ujjwalnur, Thank you for reaching out. I think it would be best to provide a code snipped we could run on our end that reproduces the problem you observe. Otherwise, it is difficult to find the source of the issue. |
Hi @JanuszL , Thanks for the response. Since it is a TFRecord based code, it would be futile to paste it here directly. Let me write a small snippet with non-TFRecord dataset and I will paste it here soon so you can play with it |
Yes, anything that reproduces the problem, like a toy example should do. |
Since I cannot attach Steps to run the code :-
Now I show you the contents of each YAML file config.yamlhorizontal_flip.yamlresize_exact.yamlMain script to run
|
Hi @JanuszL , When you run the above snippet , here is the error I am getting :
|
@ujjwalnur - can you upload a zip file with all the configs and the directory structure or something that will generate it? Copying them from the screenshots is not convenient. |
@JanuszL Here you go. |
As an additional context, if you comment out one of the augmentations from |
Were you able to reproduce the issue ? |
Yes, the repro works fine. Thank you for that. I will look into it later and get back to you soon. |
Hi, Any insights into this? |
@klecki can you look into it? |
Hi, maybe you didn't have a chance to look at it but I have tried other variations as well such as using classes in place of first-hand functions ( though first-hand functions are recommended ). But to no avail. It would be deeply appreciative to get some help on it whenever possible, as I have been stuck at it for a few weeks now. |
Hi @ujjwalnur, I am installing the dependencies right now to test the code and see what is happening. I will get back to you when I know more. |
@ujjwalnur
When using conditional execution, your code is transformed by the AutoGraph (a fork of TensorFlow library) before running it. That allows us to visit both branches of conditional statements to build DALI graph. AutoGraph transformation doesn't work well with modifications of state of existing objects and requires that in both branches the code defines matching DataNodes (the requirement for the graph). One would be to split the lists, and touch only the DataNodes that need to go through the conditional inside it, like here:
The other may be a bit simpler, we just make the scope of
|
Ok. I am still studying this -- the structure and data types of |
Describe the question.
I have tested with DALI 1.38 and DALI 1.39 on CUDA 12.2
I am trying to specify configurations for various augmentation operations in a hydra configuration file and then I want to construct those augmentations inside a pipeline. For instance, following is an example of a configuration file for random_horizontal_flip
The corresponding implementation is as below :
In a similar fashion I have configurations for other augmentations as well.
In my main code, I construct a pipeline as follows ( explanation follows )
So, as you can see following is the sequence of operations
MyDataset
.hydra.utils.instantiate
self._transform_data(....)
I have chosen this mechanism as it allows me to choose and modify all aspects of data augmentation using Hydra.
However, when running I often get error messages which tend to suggest that something is wrong with
dali_fn.random.uniform(...)
because whenever I use two augmentations ( e.g :-resize_exact
andhorizontal_flip
) it will always give me weird errors which will disappear when :a) Either I don't use any random.uniform function
b) Or I use only one augmentation
what is going wrong in my implementation ? Any hints ? Shouldn't NVIDIA DALI consider random.uniform nodes defined in different functions as different nodes ?
Check for duplicates
The text was updated successfully, but these errors were encountered: