SwapDo Startup Story

A message came through a friend. A team was building a meme service using deepfake face synthesis and needed a developer. The pitch was interesting — an app that synthesizes a user’s face onto GIF memes to create personalized content. In March 2021, I joined the four-person startup team.

The Service

SwapDo was a deepfake-based face synthesis meme creation service, supporting both Android and iOS.

The core feature was face synthesis. Users selected a GIF or image from the app’s content library, and their face was composited onto it to create a new meme. Synthesis ran in the background, so users could browse other content while waiting. A push notification arrived upon completion.

There was also a virtual plastic surgery feature — users could pick eyes, nose, or mouth from celebrities and composite them onto their own face. A meme world cup let users vote on themed content in tournament brackets. The community board allowed sharing created memes and commenting.

My Role

I served as team lead for four months, managing the team while developing simultaneously. By contribution, roughly 80% backend server development, 30% Android app development, and 10% synthesis technology.

Architecture

The service architecture was straightforward. Android/iOS clients called REST APIs on an Apache-based backend server. The backend was written in PHP, with data stored in MariaDB. Sentry handled error tracking.

Face synthesis ran in a separate environment. When the backend received a synthesis request, it invoked a Python script in an Anaconda virtual environment. Upon completion, the script returned the result file path to the backend.

flowchart LR
    subgraph Client
        Android
        iOS
    end

    Client -- "Error logs" --> Sentry
    Client -- "Request (CRUD)" --> Backend
    Backend -- "Response (JSON)" --> Client

    Backend["Apache\nBackend Server\n(PHP)"]

    Backend -- "Data R/W" --> MariaDB[(MariaDB)]
    Backend -- "Synthesis request" --> Anaconda["Anaconda\nVirtual Env\n(Python)"]
    Anaconda -- "Result path" --> Backend
    Anaconda -- "Synthesis result" --> Storage[("File\nStorage")]

The synthesis pipeline involved multiple steps: recognizing the user’s face, extracting facial landmarks, pulling frame data from the GIF content, performing 3D modeling for face synthesis, refining boundaries and skin tone, compositing frame by frame, and encoding the result back into GIF format. Libraries included OpenCV, Dlib, and FaceAlignment.

flowchart LR
    A["Face\nrecognition"] --> B["Landmark\nextraction"]
    B --> C["GIF frame\nextraction"]
    C --> D["3D modeling &\nface synthesis"]
    D --> E["Boundary &\nskin tone\nrefinement"]
    E --> F["Per-frame\ncompositing"]
    F --> G["GIF\nencoding"]
    G --> H["Result\ndelivery"]

Face recognition	Landmark extraction	3D modeling	Face synthesis	Result

Technical Contributions

Backend Refactoring

When I joined, the backend code had all logic in a single file. I introduced the MVC pattern and restructured the code with OOP principles. Separating request handling, business logic, and data access into distinct layers improved code readability and reduced response time by about 10%.

Android Infinite Scroll

The content list scroll performance was poor — noticeable stuttering during scrolling. I improved the infinite scroll logic, achieving roughly 60% faster scroll speed. The gains came from using Glide for image loading and refining RecyclerView’s recycling logic.

Asynchronous Synthesis Requests

Face synthesis required server processing time. The app could not freeze while users waited for results. I used Android’s Service component to handle synthesis requests asynchronously. While synthesis ran in the background, users could explore other content. FCM push notifications informed them when results were ready.

Retrospective

This was my first time joining a startup team. Rather than building to a handed-down spec, I was shaping what the product should be while writing the code. Leading the team was also a first. Balancing development and team management was not easy, but I came to see the product from a broader perspective.

Technically, I gained experience in PHP-based REST API design, Android app performance optimization, and background processing patterns. Working across backend and mobile in a small team built an intuition for understanding end-to-end service flows.

In July 2021, the project wrapped up naturally. Five months that started with a single message from a friend. Looking back, the experience of building a product together was the most valuable lesson.