Key takeaways
- MediaPipe is Google's open-source framework of ready-made, on-device models for hands, faces, poses, objects and more.
- MediaPipe Hands detects the palm, then predicts 21 3D landmarks per hand, tracking across frames.
- GPU support depends on platform: strong on Android, iOS and the web, more limited in Python on some systems.
- MediaPipe runs TensorFlow Lite (LiteRT) models; you do not need full TensorFlow to use it.
- It is Apache 2.0 licensed, so commercial use is allowed; TouchDesigner uses it through a community plugin.
MediaPipe uses the GPU where the platform supports it: on Android, iOS and in web browsers it can run models on the GPU, while in Python GPU support depends on the operating system and often defaults to the CPU. MediaPipe is Google's open-source framework for on-device vision and other ML tasks. I build browser-based vision with FaceVision, using ONNX Runtime Web, so this page covers MediaPipe's uses, MediaPipe Hands, GPU and TensorFlow questions, commercial use, Python, and TouchDesigner.
What is MediaPipe used for?
MediaPipe is used to add real-time perception to apps: hand tracking, gesture recognition, face landmarks, pose estimation, object detection, image segmentation, and text and audio classification, all running on the device without a server.
Developers use it for fitness apps that count exercise repetitions, sign language and gesture interfaces, virtual try-on, background blur in video calls, interactive art, accessibility tools and prototypes. Its Tasks API provides ready-made solutions for Android, iOS, web and Python, plus Model Maker for customising some models with your own data. It also includes an LLM inference API for running small language models on devices. On-device processing means low latency and better privacy, since images need not leave the device.
What is MediaPipe Hands?
MediaPipe Hands, now the Hand Landmarker task, is a model that detects hands in images or video and outputs 21 three-dimensional landmarks per hand, marking the wrist, knuckles and fingertips, plus whether each is a left or right hand.
Applications use these landmarks to recognise gestures, count raised fingers, control interfaces with pinches, track sign language handshapes, or drive animations. A related Gesture Recognizer task adds classification of common gestures, such as thumbs up or open palm, and can be customised. It runs in real time on phones and laptops, which made it one of MediaPipe's most popular solutions for creative coding and accessibility projects.
How does MediaPipe Hands work?
It uses two models in a pipeline: a palm detector finds hands in the full image, then a landmark model predicts 21 3D keypoints within each cropped hand region. On video, it reuses the previous frame's landmarks to crop the next frame, skipping detection while tracking holds.
Detecting palms rather than whole hands is easier because palms are rigid and roughly square. The landmark model was trained on real images plus synthetic hand renderings to learn varied poses. When tracking confidence drops, such as when a hand leaves the frame, the palm detector runs again. This design keeps it fast enough for real time on mobile devices, which the original 2020 paper demonstrated.
Does MediaPipe use the GPU?
It can. MediaPipe supports GPU acceleration on Android, iOS and the web, where tasks can use a GPU delegate through WebGL or WebGPU. In Python, GPU support depends on the platform and is not available everywhere, so many Python setups run on the CPU.
Also asked as: does mediapipe use gpu
MediaPipe's models are small and optimised, so CPU performance is often good enough for real time on modern laptops, especially for a single camera stream. GPU acceleration matters more for high resolutions, multiple models at once, or low-power devices. Check the current platform guide for your target, because GPU support has expanded across releases, and measure frame rates on the actual device you will deploy to.
How do I run MediaPipe on the GPU?
In the Tasks API, set the delegate in the base options to GPU when creating a task, for example a Hand Landmarker. On the web, pass the GPU delegate option; on Android, select GPU in the base options builder.
Also asked as: can mediapipe run on gpu
In Python, the equivalent is setting delegate to GPU in BaseOptions, on platforms where it is supported; otherwise MediaPipe reports an error or falls back to the CPU. Steps for any platform: confirm GPU support in the docs for your OS, enable the delegate, test that results match the CPU version, and measure speed. If the GPU path is unavailable, reduce input resolution or process every second frame for higher speed.
Does MediaPipe use TensorFlow?
MediaPipe runs TensorFlow Lite models, now called LiteRT, for inference, but you do not need to install full TensorFlow to use it. The MediaPipe package includes the runtime and models it needs.
Also asked as: does mediapipe.use tensorflow
Model Maker, used to customise some models, does depend on TensorFlow for training. Because models are in the TensorFlow Lite format, they are compact and optimised for mobile and edge devices. If you want to run MediaPipe models in other runtimes, you would need conversion, which is not always straightforward. For most users, pip install mediapipe and the documented model files are all that is required.
Can I use MediaPipe for commercial use?
Yes. MediaPipe is released under the Apache License 2.0, which permits commercial use, modification and distribution, as long as you include the licence and notices. The provided models come with model cards describing intended use and limitations.
Check each model card before shipping, since some describe fairness evaluations, intended use and known limitations, such as reduced accuracy in certain lighting or skin tones. If your product processes faces or bodies, consider privacy laws and consent, even though processing happens on the device. In FaceVision, I follow a similar privacy principle: process images on the device and store only embeddings, never the images themselves.
Why is MediaPipe used in Python?
MediaPipe is popular in Python because it offers ready-made, accurate models through a simple API, works with OpenCV for camera input, and lets students and researchers prototype hand, face and pose tracking in a few lines of code.
Also asked as: why mediapipe is used in python
Typical Python uses include computer vision coursework, gesture-controlled projects, fitness and posture analysis, data collection for research, and hackathon prototypes. Python is ideal for experimentation and batch processing of videos. For production apps on phones or the web, teams often move to MediaPipe's Android, iOS or JavaScript versions, which have better GPU support and run directly on users' devices.
Where is MediaPipe in TouchDesigner?
MediaPipe is not built into TouchDesigner. It is used through a community plugin, most commonly the MediaPipe TouchDesigner plugin by Torin Blankensmith, which you download from its GitHub releases and add to your project as a component.
The plugin runs MediaPipe's web version inside TouchDesigner's built-in web browser component, using the GPU, and outputs landmarks for hands, faces and poses as channels and tables you can connect to visuals. After downloading, open the example project or drag the MediaPipe component into your network, choose your camera, and enable the models you need. Check the repository for supported TouchDesigner versions and updates.
Is there a MediaPipe TouchDesigner tutorial?
Yes. The plugin's GitHub repository includes example projects and setup instructions, and its author and other artists have published video tutorials showing hand tracking, face tracking and pose-driven visuals in TouchDesigner.
Also asked as: mediapipe tutorial touchdesigner
A simple first project:
- Download the plugin release and open the included example.
- Select your webcam and enable hand tracking.
- Use the index fingertip landmark's x and y channels to move a shape.
- Map the distance between thumb and index finger to a parameter such as size or colour.
- Add smoothing with a Filter CHOP to reduce jitter.
From there, build interactive installations, performances or gesture-controlled audio visuals.
Sources
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. For campus sessions the inviting institution covers the visit end to end: travel (a First AC train ticket for journeys up to five hours, a flight for anything longer), accommodation, meals, and pickup and drop; the honorarium and workshop fees are negotiable on WhatsApp or on the call, and requests are welcome from institutions that can cover the visit this way. Email pranjulrathour41@gmail.com.
