Expunge

A crate for expunging sensitive fields.

Expunge
  1: to strike out, obliterate, or mark for deletion     
"In medieval and Renaissance manuscripts, a series of dots was used to mark mistakes or to label material that should be deleted from a text, and those deletion dots can help you remember the history of expunge. They were known as puncta delentia. The puncta part of the name derives from the Latin verb pungere, which can be translated as "to prick or sting" (and you can imagine that a scribe may have felt stung when their mistakes were so punctuated in a manuscript). Pungere is also an ancestor of expunge, as well as a parent of other dotted, pointed, or stinging terms such as punctuate, compunction, poignant, puncture, and pungent."

Source: https://www.merriam-webster.com/dictionary/expunge

About

At the core of Expunge is the Expunge trait, which is used for all transformations.

pub trait Expunge {
    fn expunge(self) -> Self
    where
        Self: Sized;
}

Other crates offer similar functionality, but either require types to be changed or make it difficult for both the expunged and unexpunged data being used at runtime.

This crate provides a proc_macro that derives the Expunge trait for the given type. When the Expunge::expunge method is called, sensitive fields are transformed/redacted.

All fields are transformed unless annotated with #[expunge(skip)]
The Expunge macro first looks for transformations declared on field/struct attributes i.e. as or with. If these aren't set then Expunge macro will use the Expunge::expunge implementation for the type.
A default implementation for the Expunge trait is provided for primitive types and common container types. These will be expunged as their default values, unless otherwise specified.

Since expunge doesn't require types to be changed, migrating to this crate should be completely frictionless.

Similar crates

secrecy: Prevents secrets being logged/serialized by wrapping them in a Secret<T> type
veil: A proc_macro similar to this crate to implement expunged std::fmt::Debug and/or std::fmt::Display
redact: Similar to secrecy, but without the memory zeroizing
redacted: Wrappers to control debug formatting of potentially sensitive byte arrays

Comparison

crate	proc_macro	implements Debug	serde support	toggle on/off at runtime	uses original types	slog support
secrecy	✘	✔	✔	✘	✘	✘
redact	✘	✔	✔	✘	✘	✘
veil	✔	✔	✘	✘	✘	✘
redacted	✘	✔	✘	✘	✘	✘
expunge	✔	✔	✔	✔	✔	✔

Usage

Basic usage

use expunge::Expunge;
use serde::{Serialize, Deserialize};

#[derive(Clone, Serialize, Deserialize, Expunge)]
struct User {
  #[expunge(skip)] // skipped fields are not transformed
  id: i64,
  #[expunge(as = "Randy".to_string())]
  first_name: String,
  #[expunge(as = "Lahey".to_string())]
  last_name: String,
  #[expunge(with = sha256::digest)]
  date_of_birth: String,
  latitude: f64,
  longitude: f64,
  #[expunge(as = "<expunged>".to_string(), zeroize)]
  password_hash: String,
}

let user = User{
  id: 101,
  first_name: "Ricky".to_string(),
  last_name: "LaFleur".to_string(),
  date_of_birth: "02/02/1960".to_string(),
  latitude: 45.0778,
  longitude: 63.546,
  password_hash: "2f089e52def4cec8b911883fecdd6d8febe9c9f362d15e3e33feb2c12f07ccc1".to_string(),
};

let expunged_user = user.expunge();

let output = serde_json::to_string_pretty(&expunged_user).expect("should serialize");

assert_eq!(r#"{
  "id": 101,
  "first_name": "Randy",
  "last_name": "Lahey",
  "date_of_birth": "eeb98c815ae11240b563892c52c8735472bb8259e9a6477e179a9ea26e7a695a",
  "latitude": 0.0,
  "longitude": 0.0,
  "password_hash": "<expunged>"
}"#,
  output,
)

Attributes

Container attributes (attributes that apply to a struct or enum declaration)
Field & variant attributes (attributes that can be applied to a struct field, enum variant or field in an enum variant)

Container attributes

attributes that apply to a struct or enum declaration

`as`

Provide a value that all the fields should be set to when expunged. e.g. Default::default() or "<expunged>".to_string()

Example:

In this example, all fields will be replaced with the string "<redacted>" when expunged.

use expunge::Expunge;

#[derive(Expunge)]
#[expunge(as = "<redacted>".to_string())]
struct ConnectionInfo {
  username: String,
  password: String,
  host: String,
}

`default`

Shorthand for as = Default::default(). All fields will be expunged using their Default::default() implementations.

Example:

use expunge::Expunge;

#[derive(Default)]
struct Location(f64, f64);

#[derive(Expunge)]
#[expunge(default)]
struct UserData {
  username: String,
  password: String,
  location: Location,
}

`with`

Expunge all fields using this function.

It must return the same type as it takes. e.g. hash a String with sha256::digest.

If you own the type, then could also implement Expunge directly. Using with, however, allows you to use different transformations for different fields of the same type.

Example:

In this example, fields will be replaced with their sha256 hashes.

use expunge::Expunge;

#[derive(Expunge)]
#[expunge(with = sha256::digest)]
struct Credential {
  username: String,
  private_key: String,
}

`allow_debug`

By default, expunge provides its own Debug implementation. This attribute disables the default implementation, allowing the user to implement or derive their own.

Example:

In this example, fields will be replaced with their sha256 hashes.

use expunge::Expunge;

#[derive(Expunge)]
#[cfg_attr(test, derive(Debug), expunge(allow_debug))]
struct Credentials {
    username: String,
    private_key: String,
}

Field attributes

attributes that can be applied to a struct field, enum variant or field in an enum variant

`as`

Provide a value that the given field/variant should be set to when expunged. e.g. "<expunged>".to_string()

use expunge::Expunge;

#[derive(Expunge)]
#[expunge(as = "<redacted>".to_string())]
struct ConnectionInfo {
  username: String,
  password: String,
  host: String,
}

`default`

Shorthand for as = Default::default()

Example:

use expunge::Expunge;

#[derive(Default)]
struct Location(f64, f64);

#[derive(Expunge)]
struct UserData {
  username: String,
  password: String,
  #[expunge(default)]
  location: Location,
}

`with`

Expunge the field/variant using this function.

It must return the same type as it takes. e.g. hash a String with sha256::digest

If you own the type, then could also implement Expunge directly. Using with, however, allows you to use different transformations for different fields of the same type.

Example:

use expunge::Expunge;

fn redact_first_char(mut s: String) -> String {
    s.replace_range(0..1, "*");
    s
}

fn char_count_of(s: String) -> String{
    s.len().to_string()
}

#[derive(Expunge)]
#[cfg_attr(test, derive(Eq, PartialEq, Debug), expunge(allow_debug))]
struct User {
  username: String,
  #[expunge(with = char_count_of)]
  first_name: String,
  #[expunge(with = redact_first_char)]
  last_name: String,
  #[expunge(with = sha256::digest)]
  password: String,
}

#[test]
fn field_with() {
    let user = User {
        username: "some_user_123".to_string(),
        first_name: "Jane".to_string(),
        last_name: "Doe".to_string(),
        password: "password123".to_string(),
    };

    assert_eq!(User{
        username: "".to_string(),
        first_name: "4".to_string(),
        last_name: "*oe".to_string(),
        password: "ef92b778bafe771e89245b89ecbc08a44a4e166c06659911881f383d4473e94f".to_string(),
    }, user.expunge());
}

`skip`

Skips a field. Fields marked skip will be left as-is. This is useful when:

You want to preserve fields within a struct that are not sensitive
The type cannot be expunged in a meaningful way

use expunge::Expunge;

#[derive(Expunge)]
struct UserLogin {
  username: String,
  password: String,
  #[expunge(skip)]
  last_logged_in_at: i64, // the last login timestamp will be left as-is
}

#[test]
fn skip() {
    let login = UserLogin{
        username: "gamer100".to_string(),
        password: "somepassword123".to_string(),
        last_logged_in_at: 1716113380,
    };

    let expunged = login.expunge();
    assert_eq!("", expunged.username);
    assert_eq!("", expunged.password);
    assert_eq!(1716113380, expunged.last_logged_in_at);
}

`zeroize`

Zeroize memory for extra security via the secrecy & zeroize crates.

Example:

use expunge::Expunge;

#[derive(Expunge)]
struct UserLogin {
  username: String,
  #[expunge(as = "<redacted>".to_string(), zeroize)]
  password: String, // password will be scrubbed from memory after expunging
}

Logging with `slog`

Expunge provides a painless and (relatively) foolproof way to log structs that may contain sensitive fields. As long as your type implements serde::Serialize, the slog attribute will derive slog::SerdeValue. Internally the value will be expunged before logging.

Example

use expunge::Expunge;
use serde::{Serialize, Deserialize};
use slog::{info, o};
use slog::{Drain, Logger};
use std::sync::Mutex;

#[derive(Clone, Expunge, Deserialize, Serialize, PartialEq, Eq)] // must implement Serialize
#[expunge(slog)]
#[serde(rename_all = "snake_case")]
enum LocationType {
    #[expunge(as = "<expunged>".to_string())]
    City(String),
    Address {
        #[expunge(as = "line1".to_string())]
        line1: String,
        #[expunge(as = "line2".to_string())]
        line2: String,
    },
}
 
fn main() {
    let buf = vec![];
    let drain = Mutex::new(slog_json::Json::default(buf)).fuse();
    let logger = Logger::root(drain, o!());

    // Just log as is and it will be automatically expunged

    let city = LocationType::City("New York".to_string());
    info!(logger, "it should log city"; "location" => city);

    let address = LocationType::Address{
        line1: "101 Some street".to_string(),
        line2: "Some Town".to_string(),
    };
    info!(logger, "it should log address"; "location" => address);

    // {"msg":"it should log city","location":{"city":"<expunged>"},"level":"INFO","ts":"2024-02-04T12:55:28.627592Z"}
    // {"msg":"it should log address","location":{"address":{"line1":"line1","line2":"line2"}},"level":"INFO","ts":"2024-02-04T12:55:28.627627Z"}
}