It depends. At VecorWare are a bit of an extreme case in that we are inverting the relationship and making the GPU the main loop that calls out to the CPU sparingly. So in that model, yes. If your code is run in a more traditional model (CPU driving and using GPU as a coprocessor), probably not. Going across the bus dominates most workloads. That being said, the traditional wisdom is becoming less relevant as integrated memory is popping up everywhere and tech like GPUDirect exists with the right datacenter hardware.
These are the details we intend to insulate people from so they can just write code and have it run fast. There is a reason why abstractions were invented on the CPU and we think we are at that point for the GPU.
(for the datacenter folks I know hardware topology has a HUGE impact that software cannot overcome on its own in many situations)
Location: Florida on paper (U.S. citizen born & raised); Currently residing in Nicaragua
Remote: Yes
Willing to relocate: No
Technologies: Rust, TypeScript, Python, Docker, some PyTorch / Tensorflow
Résumé/CV: https://www.linkedin.com/in/jeremyharrisconsultant/
Email: jeremy.harris@zenosmosis.com
I am using ChatGPT to help me build a language that has a domain-specific use case with non-LLM models. The interpreter is written in Rust and the language resembles a mix of Lisp and SQL.
I don't want a Facebook extension to have access to arbitrary data that I copy and paste in my browser, which may be unrelated to the app I'm developing at the time.
From what I can tell, the extension doesn't even have permission to make http requests so they aren't transmitting your data anywhere. (They also explicitly stated they don't on their listing).
> Likes/dislikes are stored in local storage and compared against all stories using cosine similarity to find the most relevant stories.
You're referring to using the embeddings for cosine similarity?
I am doing something similar with stocks. Taking several decades worth of 10-Q statements for a majority of stocks and weighted ETF holdings and using an autoencoder to generate embeddings that I run cosine and euclidean algorithms on via Rust WASM.
It seems to do well for a lot of searches, though some are questionable, but I believe that I know why. I'm training some different autoencoders to give it some different perspectives.
- "Whole OS" is a standard alpine image (4MB) with just lynx installed via standard alpine package. Plus a layer for Lynx itself and entrypoint.sh script.
So a very standardized way to run it, with reusable popular base image, decent backbone for delivering it to the public, with ability to easily mirror and/or cache (done by default) each layer. Currently base Alpine has 0 known vulnerabilities, which may not be 0 tomorrow, but it's still a marvel that it ever has such low number. New versions are available instantly after developer creates new public image, without the need for maintainer of a distribution to look at it. Meanwhile your main OS can live it's own life in his own pace, without interference.
It doesn't sound scary at all, if you really have a closer look.
reply