T
@farnz beat me to it, by a hair at that. For the sake of contributing at least something to the topic, you may want to study @quinedot's guide to dyn Trait erased types and elision rules. They aren't nearly as intuitive as you might otherwise expect them to be.
T
With a bit of fiddling, I get to: pub trait Parser { type Iter<'txt> : Iterator<Item = (usize, Vec<&'txt str>)> + 'txt; fn parse<'me, 'txt>(&'me self, input : &'txt str) -> Self::Iter<'txt>; } pub struct BoxResult<P>(pub P); impl<P> Parser for BoxResult<P> where P : Parser { type Iter<'txt> = Box<dyn Iterator<Item = (usize, Vec<&'txt str>)> + 'txt>; fn parse<'me, 'txt>(&'me self, input : &'txt str) -> Self::Iter<'txt> { Box::new(self.0.parse(input)) } } This tells the compiler very explicitly…
T
I am working on Bulkparse at Bulkgrid. It’s an MIT-licensed Rust library that converts PDFs, Word documents, Excel spreadsheets, PowerPoint presentations and other popular formats into Markdown and structured JSON. The focus is fast conversion, memory efficiency and preserving document structure. We did a 200 PDF benchmark, where Bulkparse processed the whole corpus in 0.440 seconds, 40% less conversion time than the next-fastest tested configuration. It achieved the highest reading-order,…
T
Consider the trait below. A Parser is a type which can take an input string and turn it into a bunch of candidate result. Each candidate stores: (1) the byte index up until which the result was found, and (2) a number of substrings that have been captured. pub trait Parser { type Iter<'txt> : Iterator<Item = (usize, Vec<&'txt str>)>; fn parse<'me, 'txt>(&'me self, input : &'txt str) -> Self::Iter<'txt>; } It is important that a parser's result is liftetime-independent of its output: a parser…
T
Unless I'm mistaken, it looks perhaps equivalent to the following. I think it might be a bit easier to work with if you keep things flat and serialized? fn norm2(orig: &str) -> (&str, bool) { // Try to strip off one ! If we can't return. let Some(first) = orig.strip_suffix("!") else { return (orig, false); }; // Try to strip off another !, if we can't return // what we had already stripped. let Some(second) = first.strip_suffix("!") else { return (first, true); }; // Finally, check if there was…
T
Side note: Your code may panics at unwrap() if input string contains non-ASCII characters.
T
Use is_some_and. splittersp.chars().nth_back(2).is_some_and(|chr| chr != '!') Or, better, use if let to avoid double checks for the length: if let Some(chr) = splittersp.chars().nth_back(2) { if chr != '!' { &splittersp[..splittersp.len() - 1] } else { res = true; stripped } } else { &splittersp[..splittersp.len() - 1] }
T
Shipped ferrovec 0.6.0 last week and spent this week building a real app on it. ferrovec is an HNSW vector index written in Rust and compiled to WASM (77 KB, serde + postcard only, no threads, no getrandom). In the browser it persists to OPFS with a write-ahead log, so the index survives a reload or a killed tab. To test it on something real I built a Chrome extension on top of it, Ferrovec Retrace: semantic search over the pages you have read, with the index and the embedding model both inside…
T
I have the following code fn main() { let text = ".?;,!!!"; println!("{:?}", norm(text)); let text = ".?;,!!"; println!("{:?}", norm(text)); let text = ".?;,!"; println!("{:?}", norm(text)); } fn norm<'a>(splittersp: &'a str) -> (&'a str, bool) { let mut res = false; let splitters = if let Some(stripped) = splittersp.strip_suffix("!!") { if splittersp.len() > 2 { if splittersp.chars().nth_back(2).unwrap() != '!' { &splittersp[..splittersp.len() - 1] } else { res = true; stripped } } else {…
T
Hello everyone! I am a Systems Engineering student. I’ve been studying high-performance backends and got tired of how standard message queues rely on heavy multi-threading and OS locks. So, to learn how things work under the hood, I decided to build my own from scratch in Rust. It’s called Velocity-MQ. My main goals were: Thread-per-core architecture (Shared-nothing approach). Zero-GC & Lock-Free: Built using atomic operations and Acquire/Release memory semantics instead of Mutexes. io_uring:…
T
Hey y'all, daegun 2.0.0 is out. In 1.2.0 I took the rasterizers out. 2.0.0 has one again, a new one: exact where contours overlap, and the same bytes on every CPU. The calls for drawing with your own rasterizer stay, renamed: glyph_quads is now quads and colr_scene is color_scene. Also since 1.2.0: Audited end to end: fonts that could hang it, crash it or make it hold gigabytes no longer can The shaper, the TrueType interpreter and the CFF hinter rewritten from the specs, on Unicode 18 Fonts…
T
mroth: In my opinion, introducing plugins to circumvent building an artifact, is wrong. If the build system is flawed, fix the build system, don't introduce complexity by using plugins. I additionally think, compile times are not a problem. You can organize code in different crates, use compiler caches, and/or buy a bigger machine to cut build times down. Short (required) edit/compile/test cycles are a symptom of other problems that should be solved differently. Splitting a (large) binary into…
T
shard7: This becomes particularly valuable in larger projects, especially those with numerous system-level build dependencies, where rebuilding the whole project just to introduce or update a piece of functionality can become quite cumbersome. In my opinion, introducing plugins to circumvent building an artifact, is wrong. If the build system is flawed, fix the build system, don't introduce complexity by using plugins. I additionally think, compile times are not a problem. You can organize code…
T
From what I've seen in other projects, and from my own experience, out-of-process (IPC/RPC) plugins are the sane default model for a plugin system. Mostly because of those points: A crash only takes down the plugin process, never the host, and you can add sandboxing on top. Per-plugin memory and CPU limits come from the OS (cgroups and rlimits on Linux, Job Objects on Windows), and a plugin process can always be killed, so timeouts actually work. Stopping a plugin terminates its process, and…
T
I would say that, in the absence of specific requirements, plugin processes should be neither expected to persist and reconnect to a restarted host, nor supported in doing so.[1] If the plugin wants to have a running process that outlasts the host, it should manage that separately from the plugin process managed by the host; the plugin would, separately, connect to its own persistent server. (This is the same as what it would have to do anyway if plugins were loaded libraries.) that is, the…
T
I think an unpinned nightly is inadvisable. An unpinned stable is fine though. Breakage rarely happens. On CI you might still want to pin it if you deny warnings, but in that case you can pin it inside the CI config.
T
jjpe: Well that depends. In the idealized case plugins are self-repairing. If successfully designed and implemented, it would mean that a plugin crash just means it gets reloaded, in a somewhat similar fashion as procs on Erlang's BeamVM. Panicking in such a scenario is undesirable. I was talking about host crashes, not plugin crashes. If the host crashes, nothing would need the plugin anymore, so it might as well exit immediately. If the host restarts, it can restart all plugins again. If the…
T
jjpe: 1 major difference between RPC and a .so/DLL approach is that reëstablishing a connection can be problematic (what happens to the old connection? Is it cleaned up, or leaked in the core?) while a dynlib should be possible to reload without too much fuss (especially if it's reloading the same version), which can matter for a self-healing system. My lived experience is that it's the opposite way round - if there's been an issue in the plugin host, then a dylib is impossible to reload…
T
jumpnbrownweasel: Isn't the app process (that supports plugins) the orchestrator process? With RPC? Where would it even live? In the server, or a client? If it lives in a client that's a problem hopefully for an obvious reason: there are n>=1 clients, but the exact value of n is not known. If it lives in the server, then a server crash takes the entire orchestration plan down with it as well, including live information about what is currently loaded. The unfortunate solution to this that I see…
T
jjpe: But at least you don't need a separate orchestrator process to do it, you can just create Isn't the app process (that supports plugins) the orchestrator process? jjpe: if that is not being kept in sync (eg by using a monorepo with automated testing for all plugins), Couldn't a monorepo could be used to build plugins of both kinds (DLL and RPC) to avoid versioning problems? I do agree with you on the performance difference, and this should be weighed by determining what portion of the…
T
bjorn3: Makes sense to cause a panic in all plugins when the host crashes, right? Plugins would have crashed if they ran inside the host process too. Well that depends. In the idealized case plugins are self-repairing. If successfully designed and implemented, it would mean that a plugin crash just means it gets reloaded, in a somewhat similar fashion as procs on Erlang's BeamVM. Panicking in such a scenario is undesirable. But if that's not the case, then yes, panicking can make sense, but…
T
You make good points. It's just something I noticed as it's commonly overlooked, and technically a breaking change to add later. It's probably not terribly important for that particular error.
T
Thanks for all your answers! The "local-only" case was not something I had considered, but I guess using rust-toolchain.toml there can be useful. Out of curiosity, when you then update stable / nightly locally, do you go through all your local projects to make sure they work with the new toolchain[1]? And if at a later point you decide to collaborate with others on the project, or add CI on a remote repository for it (even if private), do you then pin to a specific toolchain version, or do you…
T
jjpe: The orchestration gets more involved when the realization hits that server and client need to be kept in sync, with in-general-undefined consequences when they are out of sync. If DLLs were used, I think versioning is just as difficult.
T
jjpe: The need to orchestrate multiple processes is a PITA, because it's not as simple as just starting or killing everything. It is entirely possible that the order in which processes are started and killed matters, which would then necessitate another piece of complexity in the form of an orchestrator, complete with dependency management. Doesn't this ordering issue potentially exist, whether RPC or DLLs are used?
T
10:08 AM I’ve released an alpha of Intersect Model and Intersect Text -- modeling and language tooling for native Rust and WebAssembly. Intersect Model is inspired by the Eclipse Modeling Framework (EMF) in the Java ecosystem, with concepts corresponding to Ecore, EObject, and related model services. It supports migrating existing .ecore files and provides metamodel definitions, checked model editing, validation, transactions, and persistence. Intersect Text is inspired by Xtext and uses .xtext…
T
This week will be kind of boring for me. Continuing work on my next crate. Finishing up next month's article. Moving things to my laptop so I can work remotely. I was hopeful that I could publish my next crate before the end of this week. Maybe I'll still do that, but I'll just have it in a way lesser state than what I want.
T
I tend to go slightly further than this: pub trait OsError: Error + Send + Sync + 'static; impl<Err: OsError + From<io::Error>> { // Bits that depend on `From<io::Error>` } // or when a function depends on `From<io::Error>` fn handle_io_error<E: OsError + From<io::Error>>(error: E) { … } I find this has two advantages over putting all the constraints on the trait: I can arrange my crate and module hierarchy so that io:Error and DeviceError don't have to be visible in order to use OsError at…
T
jjpe: If the RPC server dies, in general what clients do is not cleanly defined. It could hang, fail silently, or even crash. Makes sense to cause a panic in all plugins when the host crashes, right? Plugins would have crashed if they ran inside the host process too. jjpe: Also, and I'm not 100% sure why, but in the past I've had issues with Unix sockets suddenly getting broken, making for unstable connections. If that actually frequently happened, that would cause your desktop to crash…
T
If the discomfort is purely aesthetic, I'm not sure I'd be able to either help or relate. On the other hand, if it's the relaxed clauses on where (as per the #20761) that give you the ick, you can try: use std::error::Error; use std::io; // ad-hoc auto trait to enforce `T: From<Self>` pub trait FromSelf<T: Sized + From<Self>>: Sized {} impl<T, S> FromSelf<T> for S where T: From<S> {} // original example pub struct DeviceError; pub trait OsError: Error + Send + Sync + 'static + From<io::Error> +…