Rust Excellence — TTAPI Architecture

Showcasing world-class Rust systems programming: safety, performance, and maintainability.

Overview

TTAPI is a production-grade financial data platform built with Rust, demonstrating mastery of:

1. Workspace Architecture (Polars-Inspired)

Crate Structure

ttapi/
├── crates/
│   ├── ttapi-core/          # Core types, utilities, config
│   ├── ttapi-client/        # HTTP client, auth, API wrappers
│   ├── ttapi-platform/      # Business logic, data processing
│   ├── ttapi-polars/        # Polars extensions, LazyFrame utils
│   └── ttapi-onboarding/    # CLI, user-facing interface
└── src/                     # Main binary (orchestration)

Why This Matters

Compile-time optimization:

Testability:

Reusability:

Example: Dependency Isolation

// ttapi-core: No external dependencies except std
pub struct Symbol(String);
pub struct Quote { price: f64, timestamp: i64 }

// ttapi-client: Depends on reqwest, but not on business logic
pub async fn fetch_quote(symbol: &Symbol) -> Result<Quote, ApiError>;

// ttapi-platform: Depends on core + client, implements business logic
pub async fn collect_quotes(symbols: &[Symbol]) -> Result<Vec<Quote>, TTError>;

2. Async/Await Concurrency with Tokio

Work-Stealing Scheduler

M:N threading model:

Example: Parallel Symbol Collection

use tokio::sync::Semaphore;
use std::sync::Arc;

pub async fn collect_symbols_parallel(
    symbols: Vec<String>,
    client: Arc<DynamicClient>,
) -> Result<Vec<Quote>> {
    // Limit concurrent requests to 100 (backpressure)
    let semaphore = Arc::new(Semaphore::new(100));
    
    let tasks: Vec<_> = symbols
        .into_iter()
        .map(|symbol| {
            let client = Arc::clone(&client);
            let semaphore = Arc::clone(&semaphore);
            
            tokio::spawn(async move {
                // Acquire permit (blocks if 100 tasks already running)
                let _permit = semaphore.acquire().await.unwrap();
                
                // Fetch quote (permit released when _permit drops)
                client.fetch_quote(&symbol).await
            })
        })
        .collect();
    
    // Wait for all tasks to complete
    let results = futures::future::join_all(tasks).await;
    
    // Collect successful results
    results.into_iter()
        .filter_map(|r| r.ok().and_then(|q| q.ok()))
        .collect()
}

Performance:

3. Zero-Copy Data Processing with Polars

Columnar Format Advantages

Memory efficiency:

Zero-copy operations:

Example: Volatility Calculation

use polars::prelude::*;

pub fn compute_volatility(eod_data: LazyFrame) -> Result<DataFrame> {
    // Build query plan (no execution yet)
    let volatility_lf = eod_data
        .group_by(["symbol"])
        .agg([
            col("close").std(0).alias("volatility"),
            col("close").mean().alias("mean_price"),
            col("close").count().alias("num_days"),
        ])
        .filter(col("volatility").is_not_null())
        .filter(col("num_days").gt(lit(30))); // At least 30 days of data
    
    // Execute query plan (multi-threaded, optimized)
    volatility_lf.collect()
}

Performance:

4. Type-Safe Error Handling

No Panics in Production

Philosophy:

Example: Error Hierarchy

use thiserror::Error;

#[derive(Error, Debug)]
pub enum TTError {
    #[error("API error: {0}")]
    Api(#[from] ApiError),
    
    #[error("Data error: {0}")]
    Data(String),
    
    #[error("Config error: {0}")]
    Config(String),
    
    #[error("I/O error: {0}")]
    Io(#[from] std::io::Error),
}

#[derive(Error, Debug)]
pub enum ApiError {
    #[error("HTTP {status}: {message}")]
    Http { status: u16, message: String },
    
    #[error("Authentication failed: {0}")]
    Auth(String),
    
    #[error("Rate limited: retry after {retry_after}s")]
    RateLimit { retry_after: u64 },
}

Benefits:

5. Circuit Breakers & Resilience

Fault-Tolerant Systems

Circuit breaker states:

  1. Closed: Normal operation (requests pass through)
  2. Open: Too many failures (requests fail fast)
  3. Half-Open: Testing recovery (limited requests allowed)

Example: Per-Endpoint Circuit Breaker

use std::sync::Arc;
use tokio::sync::RwLock;

pub struct CircuitBreaker {
    state: Arc<RwLock<CircuitState>>,
    failure_threshold: usize,
    success_threshold: usize,
    timeout: Duration,
}

enum CircuitState {
    Closed { failures: usize },
    Open { opened_at: Instant },
    HalfOpen { successes: usize },
}

impl CircuitBreaker {
    pub async fn call<F, T>(&self, f: F) -> Result<T, CircuitError>
    where
        F: Future<Output = Result<T, ApiError>>,
    {
        // Check circuit state
        let state = self.state.read().await;
        match *state {
            CircuitState::Open { opened_at } => {
                // Circuit open: fail fast
                if opened_at.elapsed() < self.timeout {
                    return Err(CircuitError::Open);
                }
                // Timeout expired: transition to half-open
                drop(state);
                self.transition_to_half_open().await;
            }
            _ => {}
        }
        
        // Execute request
        match f.await {
            Ok(result) => {
                self.on_success().await;
                Ok(result)
            }
            Err(e) => {
                self.on_failure().await;
                Err(CircuitError::Request(e))
            }
        }
    }
}

Retry with exponential backoff:

pub async fn retry_with_backoff<F, T>(
    mut f: F,
    max_attempts: usize,
) -> Result<T, ApiError>
where
    F: FnMut() -> Future<Output = Result<T, ApiError>>,
{
    let mut attempt = 0;
    loop {
        match f().await {
            Ok(result) => return Ok(result),
            Err(e) if attempt >= max_attempts => return Err(e),
            Err(_) => {
                attempt += 1;
                // Exponential backoff with jitter: 100ms, 400ms, 800ms
                let base_delay = 100 * (1 << attempt);
                let jitter = rand::random::<u64>() % 50;
                tokio::time::sleep(Duration::from_millis(base_delay + jitter)).await;
            }
        }
    }
}

6. TTL-Based Intelligent Caching

Automatic Cache Invalidation

No manual cache management:

Example: TTL Configuration

pub struct TtlConfig {
    pub accounts: Duration,        // 1 hour
    pub symbols: Duration,          // 24 hours
    pub trading_days: Duration,     // 7 days
    pub quotes: Duration,           // 5 minutes
    pub eod_data: Duration,         // 1 day
    pub expiration_series: Duration, // 7 days
}

pub async fn load_with_ttl<T>(
    cache_path: &Path,
    ttl: Duration,
    fetch_fn: impl Future<Output = Result<T>>,
) -> Result<T> {
    // Check if cache exists and is fresh
    if let Ok(metadata) = fs::metadata(cache_path) {
        if let Ok(modified) = metadata.modified() {
            if modified.elapsed()? < ttl {
                // Cache hit: load from disk
                return load_from_cache(cache_path).await;
            }
        }
    }
    
    // Cache miss or stale: fetch fresh data
    let data = fetch_fn.await?;
    
    // Persist to cache (background task)
    persist_in_background(cache_path, &data);
    
    Ok(data)
}

Performance:

7. Dual-Format Persistence

Parquet + CSV for Analytics & Debugging

Why dual format?

Example: Background Persistence

pub fn persist_in_background(
    data_type: &str,
    data: DataFrame,
    resource_path: PathBuf,
) {
    let data_type_owned = data_type.to_owned();
    
    // Spawn background task (fire-and-forget)
    tokio::spawn(async move {
        // Save Parquet (compressed, columnar)
        let parquet_path = resource_path.with_extension("parquet");
        if let Err(e) = save_parquet(&data, &parquet_path).await {
            tracing::error!("Parquet save failed: {}", e);
        }
        
        // Save CSV (uncompressed, row-based)
        let csv_path = resource_path.with_extension("csv");
        if let Err(e) = save_csv(&data, &csv_path).await {
            tracing::error!("CSV save failed: {}", e);
        }
        
        tracing::info!("Persisted {} ({} rows)", data_type_owned, data.height());
    });
}

Performance:

8. Code Quality Standards

Rust Excellence Playbook 2025

Mandatory practices:

Example: Test Coverage

$ cargo llvm-cov --workspace --html
Finished test [unoptimized + debuginfo] target(s) in 2.34s
     Running unittests src/lib.rs (target/debug/deps/ttapi_core-...)
     Running unittests src/lib.rs (target/debug/deps/ttapi_client-...)
     Running unittests src/lib.rs (target/debug/deps/ttapi_platform-...)

Coverage: 92.4% (12,345 / 13,367 lines)

Conclusion

TTAPI demonstrates world-class Rust systems programming through:

Built with safety, performance, and maintainability as first-class concerns.