Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add support for Windows PowerShell #7702

Merged
merged 6 commits into from
May 13, 2021
Merged

add support for Windows PowerShell #7702

merged 6 commits into from
May 13, 2021

Conversation

cenit
Copy link
Collaborator

@cenit cenit commented May 12, 2021

I might have found what failed on your system @AlexeyAB . I tried build.ps1 script only on PowerShell v6 and v7, on v5 it was failing.
Is it better now? Can you please test the script as before?

@cenit
Copy link
Collaborator Author

cenit commented May 12, 2021

@AlexeyAB the improved build script should answer your two main points:

  1. it does not close if erroring out and you are not running in a shell already opened
  2. it should work also on Windows PowerShell v5 (it was broken previously on v5, sorry for not testing that old version, I tried only v6 and v7)

@AlexeyAB
Copy link
Owner

@cenit Hi, Thanks!

Yes, now the installation process goes on, but at some point the window closed and I cannot find the darknet.exe file.
I see vcpkg folder, build_release folder, DarknetConfig.cmake.in file. Total size of the entire directory ~2GB.

Log:
vcpkg-manifest-install.log
CMakeCache.txt

@cenit
Copy link
Collaborator Author

cenit commented May 12, 2021

can you please re-run build script from a powershell already opened?
i might have missed a point of failure and the script closes without stopping on the message...
vcpkg will restart from where it left over in case you fear of loosing a lot of time

@cenit
Copy link
Collaborator Author

cenit commented May 12, 2021

i also need those files from your pc

C:\Users\HP\Desktop\Projects\yolo_test2\f4e21bd86985adb5da265e493eb512abe0381e7c\darknet-f4e21bd86985adb5da265e493eb512abe0381e7c\vcpkg\buildtrees\protobuf\install-x64-windows-dbg-err.log

C:\Users\HP\Desktop\Projects\yolo_test2\f4e21bd86985adb5da265e493eb512abe0381e7c\darknet-f4e21bd86985adb5da265e493eb512abe0381e7c\vcpkg\buildtrees\protobuf\install-x64-windows-dbg-out.log

@AlexeyAB
Copy link
Owner

@cenit
install-x64-windows-dbg-err.log

install-x64-windows-dbg-out.log is empty

@cenit
Copy link
Collaborator Author

cenit commented May 13, 2021

error: Filename longer than 260 characters

please run build.ps1 from a shorter folder, maybe just c:\darknet

i will add a diagnostics at the beginning and warn users in case something like your problem happens, hinting also a solution

@cenit
Copy link
Collaborator Author

cenit commented May 13, 2021

@AlexeyAB please try the new version. It should alert you when path is too long. And as the previous one should work ok if the path is shorter (please try something c:\darknet as i was saying)

@AlexeyAB
Copy link
Owner

@cenit

Thanks, now it works well

  • without CUDA/cuDNN/OpenCV
  • with CUDA11.02 / cuDNN8.1.1 / OpenCV4.5.1

Now it gives out warning even for a not long path, which is very often used C:\Users\HP\Desktop\Projects\yolo_test3

  • Is this a ninja problem?
  • Can we fix it somehow (besides the warning)?
  • Also can we add CI or GitHub-action to test that ./build.ps1 works well with default setting/Powershell/... ?
  • Also why does it install: protobuf:x64-windows, hdf5:x64-windows, leptonica:x64-windows, tesseract:x64-windows, libwebp:x64-windows, openssl:x64-windows, ? Do we really need all this libs?

image

@AlexeyAB AlexeyAB merged commit 179be4d into AlexeyAB:master May 13, 2021
@cenit
Copy link
Collaborator Author

cenit commented May 13, 2021

@cenit

Thanks, now it works well

  • without CUDA/cuDNN/OpenCV

  • with CUDA11.02 / cuDNN8.1.1 / OpenCV4.5.1

glad to hear that :)

Now it gives out warning even for a not long path, which is very often used C:\Users\HP\Desktop\Projects\yolo_test3

i put a very conservative length. Better safe than sorry. We can test "how big" we can go without breaking everything

  • Is this a ninja problem?

this is a windows problem. Traditionally paths on windows have a maximum length of 255 chars. Windows 10 can go beyond that but to maintain compatibility it is still hidden. No ninja problem, just a windows terrible heritage

  • Can we fix it somehow (besides the warning)?

nope. you have to live with it. It's being worked on in windows, but backwards compatibility is making it very complicated

  • Also can we add CI or GitHub-action to test that ./build.ps1 works well with default setting/Powershell/... ?

CI is already using build.ps1 in all its tests. The fact is that interactivity is not permitted on CI, so build.ps1 is always called with the -DisableInteractive parameter

  • Also why does it install: protobuf:x64-windows, hdf5:x64-windows, leptonica:x64-windows, tesseract:x64-windows, libwebp:x64-windows, openssl:x64-windows, ? Do we really need all this libs?

to build opencv+ffmpeg with a reasonable amount of features you need all of those dependencies

image

@cenit cenit deleted the dev/cenit/powershellv5 branch May 13, 2021 20:42
@cenit
Copy link
Collaborator Author

cenit commented May 13, 2021

if you meant testing older powershell in CI, that's a little bit more difficult, because github actions tends to live bleeding edge and so they have PowerShell 7 on all instances...
but now we are ready to support also pwsh5, i hope it's enough without going too far in the past

@AlexeyAB
Copy link
Owner

@cenit Ok, thanks!

Also what are the main differences/similarities/advantages of VCPKG over conda / pip?
As far as I understand, conda is not just for Python (package, dependency and environment management for any language - Python, R, Ruby, Lua, Scala, Java, JavaScript, C / C ++, FORTRAN, etc.).

@cenit
Copy link
Collaborator Author

cenit commented May 13, 2021

to build opencv+ffmpeg with a reasonable amount of features you need all of those dependencies

also, we do not decide anything about those dependencies. The opencv portfile contains all those informations about how to build it properly from sources. They might change in future but we will not have to change anything on our side

@cenit
Copy link
Collaborator Author

cenit commented May 13, 2021

@cenit Ok, thanks!

Also what are the main differences/similarities/advantages of VCPKG over conda / pip?

As far as I understand, conda is not just for Python (package, dependency and environment management for any language - Python, R, Ruby, Lua, Scala, Java, JavaScript, C / C ++, FORTRAN, etc.).

conda and pip are not meant for c/c++.
vcpkg is and it's becoming a de-facto best solution for building libraries in a cross platform way.

with a single command you can install any library on any os.

we use vcpkg to serve our dependencies, but users can use vcpkg to get darknet too (in that case it is not always the latest version, it is updated whenever i do a new release there - pointing to a newer commit from your repo of course).
it's validated so it goes under some tests

by the way, why you do not use PRs yourself when pushing to your repo? i know it might sound strange to you, but please try it, you will find that it makes work smoother and easier. You can merge your own PR at the end when you feel safe about it

@AlexeyAB
Copy link
Owner

by the way, why you do not use PRs yourself when pushing to your repo? i know it might sound strange to you, but please try it, you will find that it makes work smoother and easier. You can merge your own PR at the end when you feel safe about it

Yes, I know, that the best way is to use separate branches and Merge/do PR when it is stable ) Yes, I think it can be better for public project with big community.
I do this for production, while I usually don't do it for research.

Just they said about conda https://docs.conda.io/en/latest/

Package, dependency and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN, and more.

@cenit
Copy link
Collaborator Author

cenit commented May 14, 2021

Just they said about conda https://docs.conda.io/en/latest/

Package, dependency and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN, and more.

i do not know any real world example that uses conda to guide users to build the project from sources. only vcpkg or conan (or they just give up and list you dependencies to have to build the code)
do you know any real world example?

@FleetingA
Copy link

I am personally still getting an error on Powershell v5. Would updating it help?
error

@cenit
Copy link
Collaborator Author

cenit commented May 14, 2021 •
edited
Loading

this does not look like latest commit, so yes updating is for sure beneficial.
also please post full logs from the very first line in text mode not pictures

@cenit
Copy link
Collaborator Author

cenit commented May 16, 2021

I am personally still getting an error on Powershell v5. Would updating it help?

error

can you please retry with latest version from master? we just added some debug informations to verify your problems

@cenit cenit added this to the Easy installation of the Darknet milestone Jun 3, 2021
juhong-rdv pushed a commit to juhong-rdv/darknet that referenced this pull request Dec 18, 2021
@cenit
add support for Windows PowerShell (AlexeyAB#7702)
647184a 
* add support for powershell v5

* revert opencv enabled on osx

* improve error handling in initial stages

* intercept failures also when updating darknet sources themselves

* press a key before exiting script, to preserve console host window if not running in a shell

* check path length
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Easy installation of the Darknet
Development

Successfully merging this pull request may close these issues.
3 participants
@cenit @AlexeyAB @FleetingA