Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Camera Benchmark Tool and Allow Correct Unprojection of distance_to_camera depth image #976

Open
wants to merge 44 commits into
base: main
Choose a base branch
from

Conversation

glvov-bdai
Copy link

@glvov-bdai glvov-bdai commented Sep 11, 2024

Description

This change allows a user to easily test different camera settings/configurations to see what works best for them.

This also includes a fix to be able to correctly unproject point clouds from the distance_to_camera replicator,
as the unproject_depth method assumes that data originates from distance_to_image_plane replicator and as such could previously result in distortion when previously used with the distance_to_camera replicator.
pyproject.toml

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Screenshots

saved_depth_images_timestep_7

saved_images_timestep_4

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Copy link
Collaborator

@jsmith-bdai jsmith-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with a few minor comments. Thanks for adding this useful tool 🎉

docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
source/extensions/omni.isaac.lab/docs/CHANGELOG.rst Outdated Show resolved Hide resolved
Copy link
Collaborator

@pascal-roth pascal-roth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this tool!

I would have a proposal for the general usefulness. I find it hard to estimate how many cameras I can use in my actual scene, and I basically have to go back to the same trial and error as before, even if I used the benchmark. Personally, I would expect that I pass my scene definition (including robots, terrain, and other sensors), plus the intended data types, and then the script would add the different cameras, give me the memory consumption of each one of them, and tell me how much of each kind I can use.

docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
docs/source/how-to/estimate_how_many_cameras_can_run.rst Outdated Show resolved Hide resolved
source/extensions/omni.isaac.lab/docs/CHANGELOG.rst Outdated Show resolved Hide resolved
glvov-bdai and others added 5 commits September 18, 2024 10:38
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
glvov-bdai and others added 10 commits September 18, 2024 12:04
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Co-authored-by: Pascal Roth <57946385+pascal-roth@users.noreply.github.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
@glvov-bdai
Copy link
Author

glvov-bdai commented Sep 20, 2024

@Mayankm96 @pascal-roth

Manager based vision cartpole MR at #995 uses same convert_perspective_depth_to_orthogonal_depth utility

glvov-bdai and others added 2 commits September 20, 2024 16:41
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
depth is now aliased to distance_to_image_plane

Signed-off-by: garylvov <67614381+garylvov@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants