rlmesh

RLMesh connects a model to an environment over gRPC. This crate is the Rust facade over the transport, wire, and runtime crates. RLMesh uses the same nouns and constructs in every language, following each language’s conventions. Use this crate to serve or drive environments and models from Rust.

The two roles

• Serve an environment. Implement Env (or VectorEnv) in Gymnasium shape and host it with EnvServer, or implement EnvFactory and call EnvFactoryExt::serve.

• Serve a model. Implement Policy (or write a predict closure), build a Model from it, and host it with Model::serve.

• Run a model against an env. Model::run (or run) drives episodes on the native loop and returns a RunResult; RunHooks observe it. Model::session (or session) returns a Session to drive by hand.

Use RemoteEnv when you want to step an environment server directly.

Options structs

Optional arguments use options structs with documented defaults (EnvServerOptions, ServeOptions, EnvFactoryServeOptions). Build them with ..Default::default(): by project policy appending an options field is a minor change, and a full struct literal without .. is unsupported.

Errors

Fallible operations return Result, an alias for Result<T, Error>. Each Error variant identifies an error class: Error::class reports the exact class as an ErrorClass, whose ErrorClass::is_subclass_of tests class inheritance. Display formats the error message. The two domain failures are Error::Environment (carrying an ErrorCode) and Error::Model (a failure your Policy raised, carrying a ModelErrorCode). Both carry an is_recoverable flag surfaced by Error::is_recoverable, which is false for every other variant.

Example: serve an environment

use rlmesh::prelude::*;
use rlmesh::spaces::{self, BoxOptions, Discrete, DiscreteOptions};
use rlmesh::{EnvServer, EnvServerOptions};

struct Cartpole {
    observation_space: Space,
    action_space: Space,
}

impl Env for Cartpole {
    fn observation_space(&self) -> Space { self.observation_space.clone() }
    fn action_space(&self) -> Space { self.action_space.clone() }
    fn reset(&mut self, _seed: Option<i64>, _options: Option<&InfoDict>)
        -> rlmesh::Result<(Value, InfoDict)>
    {
        Ok((self.observation_space.sample()?, InfoDict::new()))
    }
    fn step(&mut self, _action: Option<Value>)
        -> rlmesh::Result<(Value, f64, bool, bool, InfoDict)>
    {
        Ok((self.observation_space.sample()?, 1.0, false, false, InfoDict::new()))
    }
}

let env = Cartpole {
    observation_space: spaces::Box::new(-1.0, 1.0, [4], BoxOptions::default())?.into(),
    action_space: Discrete::new(2, DiscreteOptions::default())?.into(),
};
let server = EnvServer::new(
    env,
    EnvServerOptions { address: Some("tcp://127.0.0.1:0".into()), ..Default::default() },
)?;
println!("listening on {}", server.address());
server.serve()

Example: serve a model

use rlmesh::prelude::*;
use rlmesh::{Model, ModelOptions, ModelServeOptions};

/// Counts its predicts per episode in the episode's state.
#[derive(Default)]
struct Counter;

impl Policy for Counter {
    const CONTEXT: Corners = Corners::PREDICT;

    fn predict_with_context(&mut self, _observation: Value, context: &mut PredictContext<'_>)
        -> rlmesh::Result<Value>
    {
        *context.state.get_or_insert_with("predicts", || 0u64)? += 1;
        Ok(Value::Int(0))
    }
}

let model = Model::new(Counter, ModelOptions::default())?;
model.serve("tcp://127.0.0.1:50051", ModelServeOptions::default())?;
// A closure is a policy too.
let greedy = Model::new(|_observation: Value| Ok(Value::Int(0)), ModelOptions::default())?;