Limit for thread pool

[syntheticsh]

Version

└── tokio v0.2.4
└── tokio-macros v0.2.0

Platform

4.19.0-6-amd64 #1 SMP Debian 4.19.67-2+deb10u2 (2019-11-11) x86_64 GNU/Linux

Description

I'm building runtime like this

let rt = Builder::new()
            .enable_all()
            .threaded_scheduler()
            .num_threads(4)
            .build()
            .unwrap()

Then inside runtime I download a lot of images asynchronously. And then I am spawning alot of blocking tasks. To resize them.

let thumbnail = task::spawn_blocking(move || image.thumbnail(1024, 576))
        .await?;

let _ = task::spawn_blocking(move || {
        thumbnail.save_with_format(format!("/image{}.jpg.tmp", id), ImageFormat::JPEG)
    })
    .await

on the server with 64 cores it creates at least 60+ threads (can't say for sure because I didn't count exactly, but in htop I would see threads with ids from xxxx0 to xx100 taken by this threads)
This is not optimal because other programs are starving. So I was wondering is it possible to limit thread pool. Because

.num_threads(4)

doesn't do it. In version 0.1.x I could use

blocking_threads(&mut self, val: usize)

but now it's not part of api anymore.

[luben]

I think you want task::block_in_place: https://docs.rs/tokio/0.2.4/tokio/task/fn.block_in_place.html

[Matthias247]

I would also recommend looking into adding an async semaphore on the application level for limiting the amount of parallel resizes. That decouples an application limit from a pool limit, and it might also support cancellations better (tasks wouldn't all get submitted in the first place, just in order to wait there for getting executed in the pool).

Then you can do along:

let _guard = semaphore.acquire(1);
let result = task::block_in_place(move || {
    // image stuff
});

The async semaphore in Tokio doesn't support async/await semantics yet, but until it does there are also other options available in the ecosystem.

[syntheticsh]

Thanks for replys. This seems like viable workaround. However that doesn't just make for a good experience. When I explicitly set num_threads and that contract is unexpectedly broken. There should be note in docs maybe for spawn_blocking that different worker pool doesn't have the same limit as async pool. Or better, also make it possible to limit blocking pool. I belive it's viable concern and should be addressed at some point.

[syntheticsh]

Also it seems that blocking pool is limited by this. So I wonder why it is exactly that number, and if maybe it can be changed easily without any consequences.

[syntheticsh]

After reading through my dependencies more thoroughly I found rayon in jpeg_decode in image crate, so it's not tokio's problem. What was misleading is that rayon used the same name for threads as tokio.

Thanks everyone for help.