AI Work Instructions From Photos and Short Videos of the Line

Film one task at normal speed in clips of up to three minutes, add still photos of each key position, and ask an AI assistant to draft numbered steps from the footage. Then have the operator who does the job check every step on the line, mark safety-critical steps for your responsible person, and publish a version-numbered copy.

The draft is the easy part. The value is in the capture protocol and the review, because a plausible instruction that is wrong is worse than none: operators trust a printed sheet. AI saves the hours you would spend typing, formatting and photo-cropping. It does not know your machine, your tolerances or your hazards unless someone tells it, and it cannot see what is outside the camera frame.

Follow me on Instagram@sagnikteaches

Where this fits and where it does not

Work instructions describe how one task is done: set up, run, check, clean down. They differ from broader procedures, which say who does what across a process, and from machine manuals, which the supplier writes. Here the focus is the first kind, for repeatable manual tasks such as assembling a sub-component, changing over a machine, running a quality check or packing a pallet.

Connect on LinkedInSagnik Bhattacharya

A running example: a six-person metal fabrication shop

Imagine a small shop that makes steel brackets. The most experienced welder, nearing a long holiday, is the only person who knows how to set up the jig for one customer's bracket. The owner wants a work instruction before the holiday so a second welder can cover. Everything here is illustrative.

Subscribe on YouTube@sagnikteaches

The job has about 14 steps and takes 25 minutes. The plan: capture it in two sessions, draft the instruction in an hour, review it on the shop floor with the second welder following it cold, and publish version 1.0. Estimated total effort: about four hours, against roughly six hours of typing, photo editing and layout by hand.

Step 1: agree the capture protocol

A repeatable capture protocol is what makes the AI draft good. Without it, you get shaky phone footage of the wrong angle and a draft full of guesses. Fix these rules and write them on a card by the camera.

1. One task per clip, one operator per clip. Pick the person who does the job best and ask them first. Tell them the footage is to document the task, not to assess them.
2. Tripod or fixed phone mount. Hand-held footage makes it hard for the model to follow what happens. Frame the work area, the hands and the part together.
3. Talk while working. Ask the operator to say what they are doing and why, such as "I check the fixture is clean before I load the part, otherwise it won't sit flat". The spoken reason is the detail a generic draft will lack.
4. Stills at key positions. Take a photo at each step where a position, orientation or gauge reading matters. A photo carries more precision than a frame grab from video.
5. Film the error as well. Ask the operator to show one common mistake and how it looks, such as a part loaded backwards.
6. Record the parts and settings. Photograph part numbers, machine settings and tool labels in focus, since the model can read them but may misread blurry text.

Keep the capture itself short and unhurried. A 90-second clip of a task, plus ten stills and spoken commentary, usually gives enough material for a first draft. If your site has customer-confidential parts in view, check the confidentiality position before filming, and blur or crop what the draft does not need.

Consent and privacy on the floor

Tell operators why you are filming, who will see the footage and how long you will keep it. Keep faces out of frame where you can, which also helps the model focus on hands and parts. Delete the raw footage once the instruction is signed off, and confirm that your chosen tool does not keep or train on uploads; this guide to customer data in ChatGPT covers the settings worth checking before any upload.

Step 2: get a transcript and a first draft

Use an assistant that accepts video or audio, or transcribe the spoken commentary first and paste it in alongside the photos. Gemini's app, for example, accepts video uploads; its help page puts the total video length at up to five minutes per prompt, or up to an hour on the paid Google AI Pro or Ultra plans (checked November 2026), which suits the short clips the capture card asks for. Give the model the structure you want rather than letting it choose. A fixed template produces consistent instructions across the shop and makes review faster.

You are helping me write a work instruction for a bench task in a small fabrication shop. I have attached a transcript of the operator talking through the task and 12 numbered photos.

Write a draft in this format:
- Title, purpose (one sentence), parts and tools list (only items named in the transcript or visible in the photos)
- Numbered steps, each one action in the imperative, max 25 words, with the photo number it refers to
- A "check" line after any step where the operator described a check
- A list headed "Questions for the operator" for anything unclear, missing or inferred

Rules: do not invent torque values, dimensions, part numbers or settings. If the transcript does not state one, write [TO CONFIRM]. Do not write safety instructions beyond repeating what the operator said; list any step that looks hazardous under "Flag for safety review".

The "questions" and "flag" sections matter most. They turn the model's uncertainty into a to-do list, rather than leaving it hidden inside confident prose.

A sample of the draft (illustrative)

For the bracket jig, a realistic first draft contains lines like these:

3. Wipe the jig base plate and locating pins with a dry cloth (photo 4).
   Check: no weld spatter on the pins.
4. Place the left side plate against the stop block with the cut-out facing the operator (photo 5).
5. Clamp the plate with the toggle clamp [TO CONFIRM: clamp position]
Questions for the operator:
- Is the clamp always at position 2 or does it change with plate thickness?
- At step 9 you said "tack it from the inside". How many tacks, and where?
Flag for safety review: steps 8 to 11 (welding and grinding).

Each [TO CONFIRM] is a placeholder the operator fills in. That is the system working: a draft that says what it does not know is far safer than one that guesses.

Step 3: the operator review loop

Printing the draft and walking the task is the review. Hand the draft to the operator who did the capture, then to a second person who has never done the job. The second person follows the sheet exactly, with the first watching and saying nothing unless there is a hazard. Every point where the second person hesitates, asks a question or does something different is a defect in the instruction.

Keep a simple review sheet with three columns: step number, what went wrong or was unclear, and the fix. For the bracket jig (illustrative), the walk-through might turn up five issues:

• Step 4: "cut-out facing the operator" was ambiguous, because the plate has two cut-outs. Fix: add an arrow in photo 5 and the phrase "larger cut-out".
• Step 5: clamp position missing. Fix: operator confirms position 2 for 6 mm plate, position 3 for 8 mm.
• Step 9: number of tacks not stated. Fix: four tacks, two each side, positions marked in a new photo.
• Step 12: "check the fit" is vague. Fix: reference the go/no-go gauge by its number.
• Missing step: let the jig cool before unloading. Fix: add as step 13 with the time the operator uses.

Feed the corrections back to the assistant and ask for a clean second draft. Then check that nothing else changed: models sometimes tidy wording in ways that alter meaning. Compare the two versions side by side, or ask the model to list every difference it made, and confirm each one.

Step 4: safety-critical steps stay human

This is the one section where AI should be a typist and nothing more. Hazard controls such as isolation, guarding, personal protective equipment, lifting limits and emergency actions must be written or approved by the person responsible for safety on your site, using your risk assessment and the machine supplier's documents. A model working from a video cannot see an unguarded nip point behind the operator, cannot know that a guard has been removed for filming, and may describe a shortcut as normal practice because it saw one.

A practical way to handle it: have the draft mark every hazardous step, then have your responsible person write the control wording by hand, add a visible warning symbol to the layout, and sign the review block. Where an instruction contradicts the machine manual or your risk assessment, the manual and assessment win. If you are unsure whether a step counts as hazardous, treat it as one.

Also watch for the filmed-shortcut problem. Experienced operators often skip a step they have done a thousand times, such as checking a guard interlock. If the footage shows the skip, the draft may leave it out. Ask the assistant at the end of the draft: "list standard checks a task like this usually includes that I have not mentioned", and use the answer as a prompt for the operator and your safety lead, not as a source of truth.

Step 5: format for the floor

Instructions are used standing up, with gloves, by someone glancing for ten seconds. Format accordingly. One page per task where possible, large photos beside the step, one action per step, and warnings before the step they apply to, not after. Put the part number and revision at the top and a QR code or short link to the live version if you use screens.

Ask the assistant to rewrite the long-hand steps into short imperative lines, then check each against the photo. A realistic before and after: "The operator should ensure that the jig is properly cleaned prior to loading the first component" becomes "Wipe the jig base and pins. No spatter on the pins." The second can be read at a glance. Print a test copy and read it from the distance at which it will be used.

Choosing where it lives

Whatever tool holds the instructions, the rule is one live copy per task and no stray older printouts. A shared drive folder with locked read-only PDFs works for a small shop; a document-control module or a process-capture tool suits larger ones. If your ERP or quality system already holds documents, put them there, and see whether AI can work with the tools you already use before you buy another.

Step 6: version control that operators actually follow

An instruction is a controlled document: the version on the wall must be the current one. Keep it simple.

1. Number every version: 1.0 for the first approved copy, 1.1 for a small correction, 2.0 for a changed method.
2. Put a revision block on the page: version, date, what changed in one line, author, approver.
3. One folder, read-only PDFs, with old versions moved to an archive folder, never deleted.
4. Replace printouts on the day of release, and remove the old sheet in the same visit.
5. Review on a schedule: every 12 months, and whenever the machine, material or customer requirement changes.

AI helps with the paperwork around this. Ask it to draft the revision-block line from a list of changes ("Step 5 clamp position added; step 13 added"), and to produce a short change note for the team briefing. It should not decide what counts as a method change; that is a human call because it triggers retraining.

Five more examples across a small plant

A packing instruction with a mistake photo

A pallet-packing task fails when boxes are stacked label-inwards. Include a photo of the wrong stack with a red cross beside the right one. Ask the model to caption both from the operator's commentary, and check the captions match the image.

A quality check with limits

For a visual inspection, capture ten photos each of good and bad parts under the actual lighting. The draft can describe the defect types, but the accept and reject limits must come from your specification or customer drawing; mark them [TO CONFIRM] until a quality person fills them in.

A changeover on a press

Changeovers have an order that matters. Film the whole sequence, and ask the assistant to produce a numbered list plus a table of settings before and after. The lock-out and release steps go in by the safety lead, in their own words.

Onboarding a new starter

Use the finished instruction as a training aid: the trainee reads the sheet, then does the task with a buddy, then signs the practice log. Ask the assistant for five short check-questions from the sheet, and have a supervisor approve them before use.

A cleaning and end-of-shift routine

Low-risk, high-repeat tasks such as end-of-shift cleaning are a good first project, because mistakes are cheap and the sheet gets used daily. It builds the team's confidence in the method before you tackle the welding jig.

A mistake that reached the floor

An illustrative failure shows why the review loop is not optional. A shop captured a deburring task and the draft said "remove the burr with the file, working away from the body". The operator in the video did exactly that. What the video did not show was that the operator always wore cut-resistant gloves, never mentioned in the commentary because it was habit. The sheet went out without a glove line, a new starter followed it bare-handed and cut a finger. The fix afterwards was twofold: the capture card gained a line, "say out loud every item of protective equipment you put on", and the draft prompt gained a rule to list personal protective equipment under the safety flag for the responsible person to confirm.

The wider lesson is that experts leave out the habitual. Anything the operator does without thinking is exactly what a draft from footage will omit, so ask directly: "what do you do before you start that you would never think to mention?"

A before and after: an unclear step

Here is a step as a first draft might word it from a transcript: "Next, the operator takes the part and puts it in the fixture and then checks that it is correct before continuing." Almost every word is vague: which part, which fixture, what "correct" means. After review, with the operator's detail added, it becomes two steps:

6. Load the bracket into the fixture with the stamped arrow pointing up (photo 7).
7. Check: the bracket sits flat and both locating pins are inside the holes. If it rocks, stop and call the supervisor.

Each step now has one action, a visible position and a stop condition. When you review, test every step with three questions: could a new starter do it from this line alone, is there a photo for it, and is it clear what to do if it does not go as described?

A filled-in capture card

Print this card and keep it with the camera. A completed version for the bracket jig (illustrative):

Task: Set up jig for bracket B-series, left side
Operator: [name], most experienced on this job
Date filmed: [date]  Clip numbers: 1 to 3
Stills taken: 12 (setup, clamp positions, gauge, finished part)
Said out loud: PPE worn, why the base is wiped first, how to spot a bad tack
Mistake filmed: plate loaded with cut-out reversed
Parts/settings in focus: part number plate, clamp position scale
Confidentiality checked: customer drawing not in frame
Responsible person to review: [name]

The card takes two minutes and prevents the main failure of this method: a draft based on incomplete material that nobody realises is incomplete.

What it costs and what it saves

An illustrative sum for the bracket jig. Writing the instruction by hand: about six hours (typing, photo editing and layout), at a supervisor's loaded cost of $30 an hour, so $180. With AI: roughly three hours of capture and review plus about an hour of drafting and formatting, so four hours or $120, plus the cost of an assistant plan you may already hold. The saving on one instruction is about $60. For forty tasks, that is $2,400, before counting the more important gain: the knowledge is written down before the welder goes on holiday.

Keep the bigger numbers honest. The real saving is cover and consistency, not the typing time alone. See how to calculate AI ROI for a method that includes review time and does not flatter the result.

How to check it worked

Three simple measures tell you whether the instructions are useful. First, a cold-start test: a competent colleague who has never done the task follows the sheet and completes it correctly. Second, a count of questions: how many times a new starter interrupts a supervisor on that task in the first week, which should fall. Third, defects or rework on that task over a month, compared with the month before. If the numbers do not move, look at the format and where the sheet is placed before blaming the content.

Run the first version on one shift before rolling it out, in the way described in piloting AI in shadow mode, and keep the old method available until the new sheet has survived a week. When the first task works, pick the next one using a simple frequency-times-risk ranking, and keep the same capture card, prompt and review sheet for every task. That sameness is what lets the method scale across the shop. If you would like someone to set up the capture and review routine with you, that is part of what my AI implementation consultation covers.

Questions about AI work instructions

Can AI write a safe work instruction on its own?

No. It can draft the sequence from what it sees and hears, but it cannot know about hazards outside the frame, guard settings, torque specs or your site's rules. Every safety-critical step must come from a person who knows the machine, and the finished instruction needs sign-off from your responsible person before anyone relies on it.

Which AI tools can turn a video into steps?

Several assistants accept video or photo uploads and can describe steps, and process-capture tools such as Scribe record on-screen workflows into guides. Capability and limits change quickly, so test with one real task first. Physical line work usually needs a general assistant plus your own photos, since screen-recording tools mostly cover software steps.

How long should a video capture be?

Keep each clip to one task of about 30 seconds to three minutes. Short clips are easier to check against the draft, and an error in one step does not contaminate the whole procedure. Film the task as the operator does it every day, at normal speed, then once slowly if a step is hard to see.

Is it safe to upload factory footage to a chatbot?

Check first. Footage can show proprietary processes, customer parts, people's faces and screen content. Read the tool's data-use terms, use a business plan with training switched off where one exists, crop or blur what you do not need, and get operators' agreement before filming them.

Further reads

• How to Set Up Human Review for AI Work Without Slowing Down — Set up the review step that keeps drafts honest.
• How to Scale AI From One Workflow to the Whole Business — Roll the method out to the next task without chaos.
• How to Check AI Is Doing Good Work, Not Just Fast Work — A simple routine for checking AI output against the real task.
• Customer-Facing or Back-Office: Where Should AI Go First? — Decide where AI belongs first in a small business.
• How to Brief a Developer on a Custom AI Workflow You Need — If you later want the capture routine built into your systems.
• ISO 9001 Documentation: Hire a Consultant or Draft It With AI? — Which ISO 9001 jobs AI can draft for a small manufacturer, which ones need a consultant, and a costed worked example of the split.
• 5S Audit Checklist: Build One for Your Shop Floor With AI — An area-by-area 5S scoring sheet drafted with AI from photos and notes, scored monthly by a supervisor, with AI turning the scores into a one-page trend.
• AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.

Sources: Gemini Apps Help on uploading videos and Scribe's product pages on workflow capture, checked November 2026; general document-control practice. Check each tool's current upload limits and data terms before use.