Posts from this year

UUIDv7 TypeIDs in Diesel

In a project I'm putting together, I made the choice to utilize UUIDs as identifiers early on. PostgreSQL has strong support for them, and if you're using UUIDv7, you avoid some of the issues that existed with earlier versions regarding indexing and sortability. In representing these IDs, it's common nowadays to want to utilize "Stripe-style" identifiers, something akin to:

// {slug}_{base32encodedUUID}
user_1j2d6y6h2f29awcs1fz7axq8s

On the Postgres side, as of writing this you'll need to add and load an extension for generating UUIDv7 identifiers - or you can search around for a (potentially slower) PSQL creation function.

TypeIDs in Diesel

When representing these identifiers in diesel backed types, I wanted a type that wrapped uuid::Uuid and was specific to the type and necessary {slug}_ prefix. I came up with the following solution:

/// A generic struct that we can implement the prefix over.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct UserIDPrefix;

impl TypedIDPrefix for UserIDPrefix {
    /// The prefix that this should use for public representation.
    const PREFIX: &'static str = "user";
}

/// Export the complete type for use everywhere else.
pub type UserID = TypedID<UserIDPrefix>;

Your model or struct might then look as simple as:

#[derive(Clone, Identifiable, Queryable, Selectable)]
#[diesel(table_name = crate::schema::users)]
pub struct User {
    pub id: UserID,
    pub name: String,
    pub email: String,
    // etc...
}

TypedID itself is a type that transparently handles the following scenarios:

• Loading UUIDs from Postgres via Diesel
• Writing the UUID back to Postgres, or using it in queries
• Serializing the value in "public" form, with the {slug_ base32encodedUUID} format
• Deserializing the value from the "public" form into the underlying uuid::Uuid.

With the serde implementations, you can also use this type in HTTP handlers when using frameworks like axum.

Note that this also uses the fast32 crate for base32 conversion. You can, of course, substitute whatever you'd like here.

use std::fmt;
use std::hash::Hash;
use std::io::Write;
use std::marker::PhantomData;
use std::ops::Deref;

use diesel::deserialize::FromSql;
use diesel::pg::{Pg, PgValue};
use diesel::serialize::{IsNull, Output, ToSql};
use diesel::sql_types::Uuid as PgUuid;
use diesel::*;
use fast32::base32::CROCKFORD_LOWER;
use serde::{Serialize, Serializer};
use serde::de::{self, Deserialize, Deserializer, Unexpected};
use uuid::Uuid;

/// A trait for denoting the prefix that a `TypedID<>`
/// should use when displaying/formatting.
pub trait TypedIDPrefix: Clone + Copy + fmt::Debug {
    /// The prefix to use in IDs, 4 characters or less.
    ///
    /// (Ideally 4 characters)
    const PREFIX: &'static str;
}

/// A `TypedID` is made up of two parts:
///
/// - A `prefix`, a 4-character `&'static str` that's prepended
/// - A (base32 encoded) `uuid` (v7) that forms the back half of the ID
///
/// The ID effectively ends up looking like (e.g):
///
/// ```no_run
/// user_3LKQhvGUcADgqoEM3bh6psl
/// ```
#[derive(Clone, Copy, Debug, PartialEq, FromSqlRow, AsExpression, Eq)]
#[diesel(sql_type = PgUuid)]
pub struct TypedID<PrefixType>(PhantomData<PrefixType>, Uuid);

impl<PrefixType> fmt::Display for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Formats the value.
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        write!(f, "{}_{}", PrefixType::PREFIX, CROCKFORD_LOWER.encode_uuid(self.1))
    }
}

impl<PrefixType> Hash for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Enables using this type as a hashable value.
    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
        self.0.hash(PrefixType::PREFIX);
        self.1.hash(state);
    }
}

impl<PrefixType> Serialize for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Serializes the identifier as a String, in the form of
    /// `{prefix}_{base32encodeduuidv7}`.
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: Serializer,
    {
        let id = self.to_string();
        serializer.serialize_str(&id)
    }
}

impl<'de, PrefixType> Deserialize<'de> for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Attempts to deserialize the value. This checks the prefix on
    /// an ID string before attempting to deserialize the actual UUID portion.
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        // @TODO: Could this ever be &str?
        let value: String = Deserialize::deserialize(deserializer)?;

        for (index, part) in value.split("_").enumerate() {
            if index == 0 {
                if !part.starts_with(PrefixType::PREFIX) {
                    return Err(de::Error::invalid_value(
                        Unexpected::Str(&value),
                        &PrefixType::PREFIX
                    ));
                }

                continue;
            }

            return match CROCKFORD_LOWER.decode_uuid_str(part) {
                Ok(id) => Ok(Self(PhantomData, id)),
                Err(error) => Err(de::Error::custom(error))
            };
        }

        Err(de::Error::invalid_value(Unexpected::Str(&value), &PrefixType::PREFIX))
    }
}

impl<PrefixType> ToSql<PgUuid, Pg> for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Writes the underlying uuid bytes to the output.
    fn to_sql<'b>(&'b self, out: &mut Output<'b, '_, Pg>) -> serialize::Result {
        out.write_all(self.1.as_bytes())
            .map(|_| IsNull::No)
            .map_err(Into::into)
    }
}

impl<PrefixType> FromSql<PgUuid, Pg> for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Convert from the SQL byte payload into our custom type.
    fn from_sql(bytes: PgValue<'_>) -> deserialize::Result<Self> {
        match Uuid::from_slice(bytes.as_bytes()) {
            Ok(id) => Ok(Self(PhantomData, id)),

            // For whatever reason, Diesel is able to just call `Into::into`
            // for this error - but doing that here fails?
            //
            // @TODO: Probably look into this at some point.
            Err(error) => Err(error.to_string().into())
        }
    }
}

impl<PrefixType> Deref for TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    type Target = Uuid;

    fn deref(&self) -> &Self::Target {
        &self.1
    }
}

impl<PrefixType> TypedID<PrefixType>
where
    PrefixType: TypedIDPrefix,
{
    /// Allocates and returns a new `String` in the
    /// `{prefix}_{base32encodeduuidv7}` format.
    pub fn to_string(&self) -> String {
        format!("{}", self)
    }

    /// Returns a copy of the underlying `Uuid`.
    pub fn uuid(&self) -> Uuid {
        self.1
    }
}

ORM-ish Diesel Helpers

In the Rust ecosystem, one of the more popular methods of interfacing with a database is diesel. It's not an ORM, but it has some qualities that can make it feel like one.

I'm working on a project where I wanted to add an admin interface for records that are more or less flat objects. Admin interfaces tend to very boilerplate-heavy, and this was no exception - so I wound up cobbling together a trait that enabled a slightly cleaner import and query structure for fetching and deleting entities.

It's not a solution for every use case, and you probably don't want to blanket-apply this in all scenarios - but I've found it more handy than I originally figured it'd be and so decided I'd dump it here in case anyone else is looking (especially since it seems people have been poking at this topic around the internet for a few years now).

As a bonus, jump to the next block for an async implementation. Hope it helps!

Sync Helpers

use diesel::{Connection, OptionalExtension};
use diesel::associations::HasTable;
use diesel::dsl::{Find, Limit};
use diesel::query_builder::{DeleteStatement, IntoUpdateTarget};
use diesel::query_dsl::{LoadQuery, RunQueryDsl};
use diesel::query_dsl::methods::{FindDsl, LimitDsl};

/// A type alias just to make the trait bounds easier to read.
type DeleteFindStatement<F> =
    DeleteStatement<<F as HasTable>::Table, <F as IntoUpdateTarget>::WhereClause>;

/// A trait that contains default implementations for common
/// routines that we call very often. As long as a type derives
/// `diesel::HasTable`, the methods on this trait will "just work"
/// for certain common operations that we wind up polluting everywhere
/// else. `HasTable` is generally derived on the core types that point
/// to tables, so this comes "for free" in most cases.
pub trait GenericModelMethods: HasTable {
    /// Given an entity primary key, attempts to locate and return a record from 
    /// the database.
    ///
    /// If no entity is found, this will return `Ok(None)` - it's effectively
    /// just masking the need to check for a diesel error type of `does not exist`
    /// or by filtering for something when we know it's either one or none.
    fn find_by_pk<'a, PK, Conn>(
        conn: &mut Conn,
        primary_key: PK,
    ) -> Result<Option<Self>, diesel::result::Error>
    where
        Self: Sized,
        Conn: Connection,
        Self::Table: FindDsl<PK>,
        <Self::Table as FindDsl<PK>>::Output: RunQueryDsl<Conn>,
        Find<Self::Table, PK>: LimitDsl,
        Limit<Find<Self::Table, PK>>: LoadQuery<'a, Conn, Self>,
    {
        let entity = FindDsl::find(Self::table(), primary_key)
            .first(conn)
            .optional()?;

        Ok(entity)
    }

    /// Attempts to delete a row for the specified `primary_key`.
    ///
    /// This will return the deleted row for any last minute usage.
    fn delete_by_pk<'a, PK, Conn>(
        conn: &mut Conn,
        primary_key: PK
    ) -> Result<Option<Self>, diesel::result::Error>
    where
        Self: Sized,
        Conn: Connection,
        Self::Table: FindDsl<PK>,
        Find<Self::Table, PK>: IntoUpdateTarget,
        DeleteFindStatement<Find<Self::Table, PK>>: LoadQuery<'a, Conn, Self>
    {
        let query = FindDsl::find(Self::table(), primary_key);

        let entity = diesel::delete(query)
            .get_result(conn)
            .optional()?;

        Ok(entity)
    }
}

///	Implement generic methods for all types that implement `HasTable`.
impl<T> GenericModelMethods for T
where
    T: HasTable,
{}

Async Helpers

use diesel::OptionalExtension;
use diesel::associations::HasTable;
use diesel::dsl::{Find, Limit};
use diesel::query_builder::{DeleteStatement, IntoUpdateTarget};
use diesel::query_dsl::methods::{FindDsl, LimitDsl};

use diesel_async::{AsyncPgConnection, RunQueryDsl};
use diesel_async::methods::LoadQuery;

/// A type alias just to make the trait bounds easier to read.
type DeleteFindStatement<F> =
    DeleteStatement<<F as HasTable>::Table, <F as IntoUpdateTarget>::WhereClause>;

/// A trait that contains default implementations for common
/// routines that we call very often. As long as a type derives
/// `diesel::HasTable`, the methods on this trait will "just work"
/// for certain common operations that we wind up polluting everywhere
/// else. `HasTable` is generally derived on the core types that point
/// to tables, so this comes "for free" in most cases.
#[allow(async_fn_in_trait)]
pub trait GenericModelMethods: HasTable {
    /// Given an entity primary key, attempts to locate and return a record from 
    /// the database.
    ///
    /// If no entity is found, this will return `Ok(None)` - it's effectively
    /// just masking the need to check for a diesel error type of `does not exist`
    /// or by filtering for something when we know it's either one or none.
    ///
    /// Note that this method cannot currently be generic over `AsyncConnection`
    /// due to a bug in lifetime inference in the Rust compiler.
    ///
    /// (Sub your connection type, basically)
    async fn find_by_pk<'a, PK>(
        conn: &mut AsyncPgConnection,
        primary_key: PK,
    ) -> Result<Option<Self>, diesel::result::Error>
    where
        Self: Sized + Send,
        Self::Table: FindDsl<PK>,
        <Self::Table as FindDsl<PK>>::Output: RunQueryDsl<AsyncPgConnection>,
        Find<Self::Table, PK>: LimitDsl,
        Limit<Find<Self::Table, PK>>: LoadQuery<'a, AsyncPgConnection, Self> + Send + 'a,
    {
        let entity = FindDsl::find(Self::table(), primary_key)
            .first(conn).await
            .optional()?;

        Ok(entity)
    }

    /// Attempts to delete a row for the specified `primary_key`.
    ///
    /// This will return the deleted row for any last minute usage.
    ///
    /// Note that this method cannot currently be generic over `AsyncConnection`
    /// due to a bug in lifetime inference in the Rust compiler.
    ///
    /// (Sub your connection type, basically)
    async fn delete_by_pk<'a, PK>(
        conn: &mut AsyncPgConnection,
        primary_key: PK
    ) -> Result<Option<Self>, diesel::result::Error>
    where
        Self: Sized + Send,
        Self::Table: FindDsl<PK>,
        Find<Self::Table, PK>: IntoUpdateTarget,
        DeleteFindStatement<Find<Self::Table, PK>>: LoadQuery<'a, AsyncPgConnection, Self> + Send + 'a,
    {
        let query = FindDsl::find(Self::table(), primary_key);

        let entity = diesel::delete(query)
            .get_result(conn).await
            .optional()?;

        Ok(entity)
    }
}

///	Implement async generic methods for all types that implement `HasTable`.
impl<T> GenericModelMethods for T
where
    T: diesel::associations::HasTable,
{}

Pagination in Async Diesel

In the Rust ecosystem, there are quite a few ways to approach interacting with Postgres. One of the older and more established libraries is diesel, which effectively gives you a Rust-wrapped path to constructing and executing queries. If it helps, you can think of it as a "light" ORM.

One of the bigger questions over the years has been whether diesel not being "async by default" is an issue. You could always wrap your queries in tasks and just await those, and that's probably fine - in fact, if I understand correctly, this is what crates.io does. However, diesel-async also exists for those who want to have a "pure" approach to the async interaction pattern. It's a smaller crate that effectively just requires importing a few extra traits to "override" the core diesel traits, and changing your connection type to be an asynchronous one. Overall, I've found it works pretty well.

One thing I did run into recently was needing some pagination for an admin interface I was throwing together. diesel actually has an example for this that works well if you're using synchronous calls or wrapping in tasks, but if you're using diesel-async, you'll need to tweak some things to make it work. I figured I'd share my changes here in case anyone needs them; yes, these could be on GitHub, but I'm in a weird phase of re-evaluating my usage of the platform, so I'm dumping it here.

You're smart, you can figure it out. The only real changes needed were importing some diesel-async items and tweaking the bounds on load_and_count_pages to accomodate the async differences. With this, you should be able to just call .paginate(page) on your queries and have mostly automatic paging. Enjoy.

use diesel::pg::Pg;
use diesel::prelude::*;
use diesel::query_builder::*;
use diesel::sql_types::BigInt;

// Import these, since we'll need them.
use diesel_async::AsyncPgConnection;
use diesel_async::methods::LoadQuery;

pub trait Paginate: Sized {
    fn paginate(self, page: i64) -> Paginated<Self>;
}

impl<T> Paginate for T {
    fn paginate(self, page: i64) -> Paginated<Self> {
        Paginated {
            query: self,
            per_page: DEFAULT_PER_PAGE,
            page,
            offset: (page - 1) * DEFAULT_PER_PAGE,
        }
    }
}

const DEFAULT_PER_PAGE: i64 = 10;

#[derive(Debug, Clone, Copy, QueryId)]
pub struct Paginated<T> {
    query: T,
    page: i64,
    per_page: i64,
    offset: i64,
}

impl<T> Paginated<T> {
    pub fn per_page(self, per_page: i64) -> Self {
        Paginated {
            per_page,
            offset: (self.page - 1) * per_page,
            ..self
        }
    }

    // Mark the bounds on `T` and `U` to ensure the async calls compile correctly,
    // and have `results` go through the async `RunQueryDsl`.
    pub async fn load_and_count_pages<'a, U>(
        self,
        conn: &mut AsyncPgConnection
    ) -> QueryResult<(Vec<U>, i64)>
    where
        T: 'a,
        U: Send + 'a,
        Self: LoadQuery<'a, AsyncPgConnection, (U, i64)>,
    {
        let per_page = self.per_page;
        let results = diesel_async::RunQueryDsl::load::<(U, i64)>(self, conn).await?;
        let total = results.first().map(|x| x.1).unwrap_or(0);
        let records = results.into_iter().map(|x| x.0).collect();
        let total_pages = (total as f64 / per_page as f64).ceil() as i64;
        Ok((records, total_pages))
    }
}

impl<T: Query> Query for Paginated<T> {
    type SqlType = (T::SqlType, BigInt);
}

impl<T> RunQueryDsl<PgConnection> for Paginated<T> {}

impl<T> QueryFragment<Pg> for Paginated<T>
where
    T: QueryFragment<Pg>,
{
    fn walk_ast<'b>(&'b self, mut out: AstPass<'_, 'b, Pg>) -> QueryResult<()> {
        out.push_sql("SELECT *, COUNT(*) OVER () FROM (");
        self.query.walk_ast(out.reborrow())?;
        out.push_sql(") t LIMIT ");
        out.push_bind_param::<BigInt, _>(&self.per_page)?;
        out.push_sql(" OFFSET ");
        out.push_bind_param::<BigInt, _>(&self.offset)?;
        Ok(())
    }
}

Plant-Based Berlin Recommendations

2023 has come and gone, and with it, a decent chunk of travel. The list below is an assortment of plant-based/vegan options in and around Berlin, Germany. I visited most of these during a recent trip, but never compiled them into a list until recently. I figure they're as good as anything for content to update this old still creaking along site - and it beats writing about tech for the millionth time.

Eat

• Dervish
  Vegan Uzbek cuisine. In a nightlife hotspot and open decently late, so you could do either lunch or dinner here. We never needed to make a reservation, and there are a number of other vegan restaurants on the stretch should they be full. It's not the best thing in the world, but it is cool just because it's pretty rare to find a vegan place that does this.
• Frea Bakery / Frea Restaurant
  Zero-waste concept, all vegan. The bakery is two or so blocks from the restaurant, both are worth going to. The bakery is great for working out of if you're in need of space. Sanctuary - a vegan italian bakery - is around the corner, but if I had to choose one to go to, it'd probably be Frea. If you want the fine dining here but can't grab a booking, Cookies Cream or Lucky Leek are good second options.
• Vöner - doner kebab
  The original "vegan kebab" and pretty much still the best. I tried quite a few alternatives before finally coming here and wish I had just come here to begin with. Some reviews say the staff are rude or whatever, but I wound up talking to them for 15 minutes and they were super chill - YMMV but I wouldn't be put off by it.
• Chay Long P-Berg / Chay Long Hasenheide
  Vegan Vietnamese with two locations. The menu is slightly different depending on which location you go to; we went to the Hasenheide location since it has a Bún Cá Hải Phòng on the menu that we hadn't seen elsewhere. I remember liking it, though my wife said it was good but not perfect. It's kind of a tough one to vegan-ize though.
• Tsu Tsu Kaarage
  We found this while walking around, pleasantly surprised since it gets close enough to what you'd want from kaarage. Small place with a few seats. There's also a pie shop around the corner from here that serves Oatly soft serve, but the name escapes me unfortunately.
• Kimchi Princess KBBQ
  KBBQ spot that - surprisingly - has two entirely vegan options. We tried the seitan one and it's decent enough, and it is grilled in front of you, but they seem to not want you to handle the grill yourself which annoys me to no end. That said, I feel like finding this kind of thing with vegan options is already hard, so it was nice to experience again. They have tables for 2 if you don't want to be at a long table with others who grill meat or anything.
• la petite véganerie - brasserie
  This one is newer and we found it while looking for vegan steak. They basically try to properly recreate a French cafe style experience and have various cheeses/steaks/etc to pull it off. If you're looking into trying some plant-based steak options that we don't have in the USA - or are hard to get over here - this place can be fun. When we went the service was slower than I can possibly put into words, but it just seemed like growing pains and the staff were really chill. You can technically walk from here to Becketts Kopf if you want cocktails after.
• Tbilisi
  Georgian restaurant that has vegan options. We wanted to go here as several friends recommended it, but it unfortunately got screwed schedule-wise. Listing it mostly on recs from others and for variety.
• Försters - Das Vegane Restaurant
  Because it's Berlin and you probably want some German food at some point. The website for this place is like... useless, but the food is decent and it's got massive portions for schnitzel. They also have options for the type of schnitzel, ranging from soy-based to... something else. I forget what. (Every list I read says to go to Vaust, but when we tried to go there they were really standoff-ish and said they weren't open anymore... well before their closed time... with no sign of an event going on. YMMV though.)

(I did want to add in some easy vegan breakfast spots here, but my two go-tos - No Milk Today and Cafe Nullpunkt - unfortunately shuttered at the end of 2023. That said, most coffee shops I've been in there have some vegan option nowadays due to how widespread the concept is.)

Drink

Hope you enjoy your clothes reaking of smoke. For some of the below, reservations are a good idea.

• Timber Doodle is a casual cocktail bar around the corner from Dervish that, if it's not too crowded, has some creative cocktails and good energy.
• Bar Becketts Kopf
  Yes, it's on the world's 50 best bars list or whatever. It's legitimately good for cocktail stuff though, and the wall art inside is... well, surreal. Pretty sure the door is unmarked.
• Buck and Breck
  Also on 50 best list but drinks are legitimately good. No sign or anything so another unmarked bar, phones banned inside.
• Velvet
  Another cocktail bar but tends to focus more on seasonal/local ingredients. Near-ish to the Chay Long Hasenheide location if you want to stack them. We wandered in on an industry night and somehow got seated, so dunno what the normal menu is - but the vibe is great.
• Protokoll
  Craft beer options in Berlin generally suck but this place is decent with variety and vibe.
• Motif
  Natural wine bar + record shop. Out of the way, but if you're in the area it's cool to know about. Should have vegan wines but you should of course ask.

Clubs

• ://about blank
  Industrial club that is larger than it seems. Indoor area with rotating genres but usually house, outdoor chill area, back building that rotates genres like 70s remixes or local artists. The back building is either really great or really "where the actual fuck am I" experience-wise, but the club overall is great albeit out of the way. They're strict about phone use, don't be a dick to the bouncers.

Shop

• Veganz grocery store.
  It's easy to find - right off a large train station - and you can't miss it. Fun to walk through if you like seeing what other vegan options are available outside the USA.