Inference on an embedded MCU (RP2040 / Raspberry Pico)

[mvniekerk]

Feature description

I'd like to run an inference model on an embedded MCU, specifically an RP2040. Like to TensorFlow Micro (https://lib.rs/crates/tfmicro).

Feature motivation

To do event sourcing remotely on electronic devices.

(Optional) Suggest a Solution

[antimora]

There is one issue relating to Atomic locking that has not been resolved yet but this was mainly with Ndarray backend. Please see: #302. It might be possible with Candle CPU backend, which I haven't tried yet. I am willing to assist if someone can work on this.

[antimora]

But generally we do support no_std and our CI is building with thumb 7m target.

[nathanielsimard]

If everything is single threaded, then atomic is a non-issue.

[mvniekerk]

Oh wow thanks all for the comments.
The RP2040 is definitely on the thumbv6m-none-eabi target, so the locking is problematic then.
I see there's a WIP PR on the locking issue for the exact target that has been closed since.

If there's some path to success on this then I'll also assist gladly to get this up.

[nathanielsimard]

@mvniekerk Have you tried running a model with the Candle CPU backend and/or the NdArray backend? If there are some errors it might help to know what is missing.

[BjornTheProgrammer]

@nathanielsimard I'm trying to do the same thing as @mvniekerk and I get the same error messages, I tried using candle and ndarray, but both of those failed.

Using this

burn = { version = "0.13.0", default-features = false, features = ["ndarray"] }

Gave the following errors

error[E0432]: unresolved import `alloc::sync`
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ndarray-0.15.6/src/lib.rs:129:12
    |
129 | use alloc::sync::Arc;
    |            ^^^^ could not find `sync` in `alloc`

error[E0432]: unresolved import `alloc::sync`
  --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ndarray-0.15.6/src/data_traits.rs:16:12
   |
16 | use alloc::sync::Arc;
   |            ^^^^ could not find `sync` in `alloc`

error[E0599]: no method named `compare_exchange_weak` found for struct `AtomicBool` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/mutex/spin.rs:182:14
    |
180 |   ...   while self
    |  _____________-
181 | | ...       .lock
182 | | ...       .compare_exchange_weak(false, true, Ordering::Acquire, Orde...
    | |           -^^^^^^^^^^^^^^^^^^^^^ method not found in `AtomicBool`
    | |___________|
    |

error[E0599]: no method named `compare_exchange` found for struct `AtomicBool` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/mutex/spin.rs:242:14
    |
240 |   ...   if self
    |  __________-
241 | | ...       .lock
242 | | ...       .compare_exchange(false, true, Ordering::Acquire, Ordering:...
    | |           -^^^^^^^^^^^^^^^^ method not found in `AtomicBool`
    | |___________|
    |

error[E0599]: no method named `compare_exchange` found for struct `core::sync::atomic::AtomicU8` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/once.rs:114:18
    |
112 |   ...   match self
    |  _____________-
113 | | ...       .0
114 | | ...       .compare_exchange(old as u8, new as u8, success, failure)
    | |           -^^^^^^^^^^^^^^^^ method not found in `AtomicU8`
    | |___________|
    |

error[E0599]: no method named `fetch_add` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:265:31
    |
265 |         let value = self.lock.fetch_add(READER, Ordering::Acquire);
    |                               ^^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_sub` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:268:23
    |
268 |             self.lock.fetch_sub(READER, Ordering::Relaxed);
    |                       ^^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_sub` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:305:23
    |
305 |             self.lock.fetch_sub(READER, Ordering::Release);
    |                       ^^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_sub` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:349:19
    |
349 |         self.lock.fetch_sub(READER, Ordering::Release);
    |                   ^^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_and` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:363:19
    |
363 |         self.lock.fetch_and(!(WRITER | UPGRADED), Ordering::Release);
    |                   ^^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_or` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:415:22
    |
415 | ...   if self.lock.fetch_or(UPGRADED, Ordering::Acquire) & (WRITER | UP...
    |                    ^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_sub` found for reference `&'rwlock AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:760:19
    |
760 |         self.lock.fetch_sub(READER, Ordering::Release);
    |                   ^^^^^^^^^ method not found in `&AtomicUsize`

error[E0599]: no method named `fetch_sub` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:770:25
    |
770 |         self.inner.lock.fetch_sub(UPGRADED, Ordering::AcqRel);
    |                         ^^^^^^^^^ method not found in `AtomicUsize`

error[E0599]: no method named `fetch_and` found for struct `AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:782:14
    |
780 | /         self.inner
781 | |             .lock
782 | |             .fetch_and(!(WRITER | UPGRADED), Ordering::Release);
    | |             -^^^^^^^^^ method not found in `AtomicUsize`
    | |_____________|
    |

error[E0599]: no method named `compare_exchange` found for reference `&AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:796:16
    |
796 |         atomic.compare_exchange(current, new, success, failure)
    |                ^^^^^^^^^^^^^^^^ method not found in `&AtomicUsize`

error[E0599]: no method named `compare_exchange_weak` found for reference `&AtomicUsize` in the current scope
   --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/spin-0.9.8/src/rwlock.rs:798:16
    |
798 |         atomic.compare_exchange_weak(current, new, success, failure)
    |                ^^^^^^^^^^^^^^^^^^^^^ method not found in `&AtomicUsize`

Using this

burn = { version = "0.13.0", default-features = false, features = ["candle"] }

I got the following errors along with hundreds of others, too much to add here.

error[E0412]: cannot find type `Result` in this scope
    --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.197/src/private/ser.rs:1258:21
     |
1258 |     fn end(self) -> Result<(), Self::Error> {
     |                     ^^^^^^ not found in this scope
     |
help: consider importing one of these items
     |
1    + use alloc::fmt::Result;
     |
1    + use core::fmt::Result;
     |
1    + use core::result::Result;
     |

error[E0425]: cannot find function, tuple struct or tuple variant `Ok` in this scope
    --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.197/src/private/ser.rs:1259:9
     |
1259 |         Ok(())
     |         ^^ not found in this scope
     |
help: consider importing this tuple variant
     |
1    + use core::result::Result::Ok;
     |

error[E0405]: cannot find trait `Sized` in this scope
    --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.197/src/private/ser.rs:1292:28
     |
1292 |     fn serialize_field<T: ?Sized>(&mut self, value: &T) -> Result<(), ...
     |                            ^^^^^ not found in this scope
     |
help: consider importing this trait
     |
1    + use core::marker::Sized;
     |

error[E0412]: cannot find type `Result` in this scope
    --> /Users/bjorn/.cargo/registry/src/index.crates.io-6f17d22bba15001f/serde-1.0.197/src/private/ser.rs:1292:60
     |
1292 | ...lf, value: &T) -> Result<(), Self::Error>
     |                      ^^^^^^ not found in this scope
     |
help: consider importing one of these items
     |
1    + use alloc::fmt::Result;
     |
1    + use core::fmt::Result;
     |
1    + use core::result::Result;
     |

I can create a repo to reproduce the errors if wanted.

[BjornTheProgrammer]

This should be fixed with the latest merge. In the examples/ directory there is an example of using inference on the rp2040.