Director of Data Engineering & AI — Tel Aviv, Israel

I build the LLM and data platforms behind the product, and I’ve led the teams that own them.

At Tastewise I built the first real data platform, then the agent stack on top of it. I tackle the hardest problems together with my team.

Portrait of Michael Ostrovsky
01 /

Experience

Reverse-chronological. Every promotion came with a harder problem to own.

  1. Now

    Director of Data Engineering & AI

    Tastewise

    Tech Lead Team Lead, Application Data & AI Team Lead Director of Data Engineering & AI

    Started on the core ETL platform, built out the company’s first proper data platform, and grew into owning the data products and the whole AI/agent layer.

    • Built and have owned the LLM pipeline behind the product’s AI features since 2023 — pull data from Elasticsearch, ground a versioned prompt in it, call the model, and turn the result into imagery with genai — with runtime JSON-schema validation that feeds errors back to the model for automatic repair, back when getting valid JSON out of an LLM was still hard.
    • Cut the wait from a ~90-second blank screen to answers that appear as they generate: moved the LLM path onto WebSocket token streaming with optimistic partial-JSON parsing, so the UI renders partial results instead of holding for the full response.
    • Added an LLM router that classifies each request and routes it to the right flow, chained dependent calls, and built the operational scaffolding around them — PromptLayer for prompt versioning (which I evaluated and introduced; still in use) and Langfuse for tracing the chains.
    • Stood up internal AI-platform infrastructure: MCP servers exposing the company’s data platform to agents, an OAuth2 AI gateway that mints per-user tokens, and a Slack knowledge-sharing agent owning the team’s LLM wiki.
    • Led an internal platform that lets anyone at the company build sub-apps inside the product — no R&D deployment or security burden. With my AI engineers I built the MCP layer giving the Lovable coding agent governed access to Tastewise: a system prompt, progressive-disclosure API tools (BM25 over our OpenAPI), a Storybook MCP for the design system, and the PromptLayer MCP — with skills delivered as markdown tool-outputs where Lovable can’t take them over MCP — all behind OAuth2 through our AI gateway.
    • Introduced Claude Code to R&D in July 2025 and drove its adoption — within a couple of months the whole org was building with it — then led the build of an internal Claude Code plugin packaging the company’s shared skills and MCP servers.
    • Moved the data platform onto Databricks and Delta Lake over 292 commits, rewriting jobs that used to run for days into PySpark on Airflow, so they finish in hours instead.
    • Built the ingestion behind 1M+ indexed restaurants & venues, unifying 10 mismatched collectors into one schema and orchestrating the jobs on Airflow with KubernetesPodOperators (EKS).
    • Kicked off the Angular 12 → React 18 migration, writing an interop layer that rendered React components inside the live Angular app for an incremental cutover — widening the hiring pool and lifting delivery velocity.
    • Started Tastewise’s design system with the designers in 2023 and built its first Storybook — later the basis for the Storybook MCP that lets the internal app-builder’s agent build with the real components.
    • Built company-wide SSO by hand with Passport.js — no managed auth provider — standing up authentication across the product’s services.
    • Led the observability overhaul, moving the platform from Logz.io to Datadog and standardizing a custom Node.js & Python logger across every service, so following a request went from grepping scattered logs to a single query — cutting bug investigations from hours to minutes.
    • Modernized the backend across two migrations: TasteGPT from Flask to FastAPI, and the Nodejs scrapers from TypeScript to a fully async Python framework.
    • Grew the team from 2 to 9 engineers across full-stack, data, AI and data science, built the interview process we still use, and took over the prior team’s ML pipelines with zero handover.
    • Python
    • Databricks
    • PySpark
    • Airflow
    • Kinesis
    • Firehose
    • Lambda
    • Pydantic
    • MCP
    • Langfuse
    • FastAPI
    • Kubernetes (EKS)
    • Datadog
    • TypeScript
  2. Full-Stack Team Leader

    Zencity

    Architected the civic-engagement (B2G) product’s survey-submission platform, and grew from developer to leading the full-stack team.

    • Cut survey-collection cost to near zero by replacing a third-party SaaS with an in-house platform — a React client plus a pipeline of event-driven Lambda microservices over SQS, fanning each submission through ingestion, score aggregation, ML entity-enrichment (BERT) and location enrichment before persistence — running across ~100 cities at thousands of submissions a day.
    • Championed and drove the Node.js codebase migration from JavaScript to TypeScript, sharply reducing production bugs, and added a schema layer over MongoDB that further hardened system stability.
    • Operated, maintained and monitored the Confluent Kafka CDC (Change Data Capture) pipeline streaming MongoDB’s oplog into Elasticsearch — thousands of events a day fanned across 50 consumer groups, partitioned by a modulo on the Mongo ObjectId.
    • Built Zencity’s first microfrontend — a host-and-remotes Webpack Module Federation architecture — upgrading Webpack and wiring the remotes to load and communicate with each other both locally and across staging and production.
    • Built device fingerprinting and throughput optimization into the submission flow, hardening it and making it faster to onboard new cities.
    • Grew and led the full-stack team from 2 to 7 engineers after being promoted from developer.
    • TypeScript
    • React
    • Node.js
    • AWS Lambda
    • SQS
    • Kafka (Confluent)
    • MongoDB
    • Elasticsearch
    • Webpack Module Federation
  3. Co-Founder & VP R&D

    Snipe

    Co-founded an esports startup and ran R&D, shipping ML-driven products for League of Legends players.

    • Built Sightstone, which predicted favorable League of Legends matchups by pre-ingesting millions of games — Spark/Scala for processing, scikit-learn for the models, MongoDB for storage.
    • Built Matchmaker, a mobile app that profiled players’ playstyle to recommend teammates and streamers, tackling game loneliness and content discovery.
    • Grew to 100k+ downloads and 10k+ weekly active users across 4 products in two years, raised a pre-seed round, and moved the company to Silicon Valley.
    • React
    • React Native
    • AWS
    • MongoDB
    • Spark (Scala)
    • scikit-learn
  4. Data Developer Team Leader

    Israeli Military Intelligence — Unit 8200

    Led a data team building tools that surfaced new insight from previously-inaccessible data.

    Israel’s elite SIGINT unit, the talent pipeline behind Check Point, Palo Alto Networks and Wiz.

    • Built tooling that made previously-inaccessible mission-critical data queryable, unlocking insights from data the organization already held.
    • Led a team of 5 data developers and analysts across seniority levels.
02 /

Selected work

Four things I built or led.

  1. 01

    Product LLM pipeline

    2023 - 2026

    The LLM layer behind the product’s AI features: pull data from Elasticsearch, ground a versioned prompt in it, call the model, and validate-and-repair the JSON. An LLM router picks the flow; responses stream back token-by-token with partial-JSON parsing; chained calls are traced in Langfuse.

    Runs the product’s AI features in production — fast first paint via streaming, every chain traced.

    • Python
    • FastAPI
    • WebSockets
    • PromptLayer
    • Langfuse
  2. 02

    Databricks lakehouse migration

    2025

    Led the move off scattered scripts that ran for days onto a Databricks and Delta Lake lakehouse: bronze/silver/gold layers, Asset Bundles, parameterized Spark jobs, multi-country fan-out.

    About 10× faster, shipped incrementally over 292 commits.

    • Databricks
    • PySpark
    • Delta Lake
    • Airflow
  3. 03

    Ingestion at 1M+ restaurants & venues

    2024

    Moved batch jobs off the ad-hoc EC2 scripts a human used to babysit onto orchestrated, containerized workloads, streaming through Kinesis, Firehose and a Lambda with Pydantic guarding against upstream schema changes.

    1M+ restaurants & venues matched and indexed, holding steady as upstream sources kept changing shape.

    • Kinesis
    • Firehose
    • Lambda
    • Pydantic
    • Kubernetes
    • Airflow
  4. 04

    Self-service app builder (Lovable + MCP)

    2026

    An internal platform letting any employee build sub-apps inside the product. I built the MCP layer that gives the Lovable coding agent governed access to Tastewise: a system prompt, progressive-disclosure API tools (BM25 over our OpenAPI), a Storybook MCP for the design system, the PromptLayer MCP, and skills as markdown tool-outputs — all behind OAuth2 through an AI gateway we built.

    Employees ship realistic apps on the company design system and data, with no R&D deployment or security overhead.

    • MCP
    • Lovable
    • BM25
    • OAuth2
    • Storybook
03 /

Stack & skills

AI & LLM Engineering

  • LLM orchestration (chaining + routing)
  • Retrieval-augmented prompting
  • MCP servers & gateways
  • Langfuse evals & observability
  • Structured-output validation & repair
  • Token streaming

Data Engineering

  • Databricks
  • Delta Lake
  • PySpark / Spark
  • Airflow (AWS MWAA)
  • Kinesis / Firehose
  • Kafka (CDC)
  • Pydantic
  • Elasticsearch
  • MongoDB
  • S3

Cloud & Infra

  • AWS
  • Kubernetes (EKS)
  • KubernetesPodOperators
  • Lambda
  • Datadog (APM/RUM)
  • Terraform

Languages & Backend

  • Python
  • TypeScript
  • Node.js
  • FastAPI
  • Flask
  • NestJS
  • Microservices
  • Auth & SSO (Passport.js, OAuth/JWT)

Frontend

  • React
  • React Native
  • Svelte
  • TypeScript
  • SCSS
  • Microfrontends (Module Federation)
  • Framework migration (Angular → React)

Leadership

  • Hiring & interview design
  • Org design
  • Mentoring
  • Technical strategy
  • Cross-functional delivery
04 /

Education

05 /

Get in touch