Ethics, Uncertainty, and the Art of Engineering

ML for Science and Engineering - Lecture 29

Tools, responsibility, and how to act when you cannot know what is right

Joseph Bakarji

You Are a Tool

Engineers build tools. That is the job.

You learn to build small tools, then bigger tools.

When you build tools for a company, in a structural sense, you are a tool. The company uses you to build its tools.

The concept of using something and being used generalizes across humans and machines.

Key insight This is not cynical. It is structural. Understanding where you sit in the chain is the first step toward acting with intention.

Means to Ends

Tools are means to ends. Trace any tool far enough and you arrive at a small set of human ends.

Hammer → Nail → Join wood → House → Shelter → Survival → Well-being
There are many tools, but the ends are few: survival, comfort, happiness, meaning. Most tools exist not merely for survival, but for well-being.

Nested Scales

You are building a screw in a very large machine.

Biology
A cell in your hand does not know why the hand is moving. A brain cell is not aware of the intentions of the whole brain.
Engineering
You build a component. It fits in a subsystem. The subsystem fits in a product. The product fits in a market. The market fits in society.
As technology scales globally, the distance between the tool you build and the purpose it serves keeps increasing. We are less and less sure what the tools we build are for.

AI Makes This Worse

AI systems are not only part of larger opaque systems, they are themselves unpredictable.

Traditional tool: You know the input-output relationship. The wrench turns the bolt.

AI tool: You release a neural network. People use it in ways you did not anticipate. The consequences branch unpredictably.

Double opacity You cannot see the larger system your tool fits into, AND you cannot predict what your tool itself will do.

The Intelligence-Predictability Tradeoff

Unpredictability is not a defect. It is what makes intelligence interesting.

A perfectly predictable tool is not creative.

If you hire someone to be creative, you must tolerate unpredictability. You do not know what ideas they will produce.

The same holds for AI. If you want it to be useful in genuinely new situations, it will surprise you.

The tradeoff
Predictable ↔ Creative
Controllable ↔ Intelligent
Safe ↔ Useful

Deterministic but Unpredictable

You already know this from this course.

Chaotic systems are deterministic. Given the same initial conditions, they produce the same trajectory. But tiny perturbations lead to vastly different outcomes.

No matter how many rules you impose on a high-dimensional system, the dimensionality defeats regulation.

From this course The Lorenz system has only 3 variables and 3 equations. It is fully deterministic. Yet it is unpredictable beyond a short horizon. Now imagine a system with billions of parameters interacting with millions of users.

The Consequence Tree

One tool, many outcomes. Click nodes to explore branches.

Click any node to expand/collapse. Color indicates value: positive to negative.

The Distribution of Outcomes

Model consequences as a probability distribution over a value spectrum.

0.3
1.0
You want to push the mean right (more good outcomes) and narrow the spread (fewer extreme bad outcomes).

We Already Manage Dangerous Tools

Every powerful technology comes with guidance. AI should be no different.

Chemicals
"Do not drink."
Warning labels, MSDS sheets, regulated concentrations.
Medicine
Dosage instructions, contraindications, clinical trials before release.
AI
???
The industry has not converged on safety standards yet.
The gap is not that AI is fundamentally different. The gap is that we have not yet built the equivalent of clinical trials, warning labels, and dosage instructions for AI systems.

Human-Centered Testing

A practical approach to managing unpredictable tools.

1. Prototype — Build a small version of the tool
2. Observe — Give it to a small group, watch how they actually use it
3. Document — Record failure modes, unintended uses, edge cases
4. Guide — Write instructions: proper use, known risks, contraindications
5. Release — Ship with the manual, not without it

Three Levels of Ethical Thinking

The type of question you ask determines which level you are working at.

Applied Ethics — Is this specific thing good or bad?
"Should civilians own guns?" "Should we deploy this algorithm?"
Normative Ethics — What theory should govern our judgments?
"Maximize happiness" (utilitarianism), "Follow universal rules" (deontology), "Cultivate virtuous character" (virtue ethics)
Meta-Ethics — Does good and bad even exist? On what basis?
"Is morality objective, subjective, or a social construction? Does it need grounding?"
Just as tools are nested inside bigger tools, ethical reasoning is nested inside deeper ethical reasoning. You will not find bedrock.

Applied Ethics: The Gun Debate

A concrete example of applied ethics with no clean resolution.

For
  • Right to self-defense
  • Protection against government overreach
  • Rural areas: help is far away
  • Constitutional tradition
Against
  • Any civilian can kill any other
  • Mass shooting statistics
  • Escalation dynamics
  • Children's access
You can argue endlessly at this level. Applied ethics without a framework is just debate. To make progress, you need to step up one level: what theory of good are you using?

Normative Ethics: Theories of Good

Each theory gives you a lens. Each lens distorts something.

Theory Core Principle Limitation
Utilitarianism Maximize collective happiness You cannot sum happiness; ignores distribution
Deontology Follow universal moral rules Rules conflict; rigid in edge cases
Virtue Ethics Cultivate virtuous character traits Who defines virtue? Culture-dependent
Divine Command Good = what the creator commands Requires shared religious framework
A theory is a model of ethics. Like scientific models, ethical theories rely on assumptions and eventually fail in some domain.

Meta-Ethics: Does Good Exist?

You have intuition that 1 + 1 = 2.
You have intuition that some things are good and some are bad.

Neither has objective grounding beyond the feeling.

Some positions:

  • Moral realism: Good and bad are objective features of reality
  • Subjectivism: "Good" means "I like it"
  • Constructivism: Morality is a social agreement, not a discovery
  • Nihilism: The value system has no grounding at all

Uncertainty All the Way Down

Here is the deeper point. It is not just ethics that is uncertain.

Your entire understanding of the world is subjective. Every model, every differential equation, every scientific framework happens inside your head.

Perhaps it is collective subjectivity. But subjective nonetheless.

If scientific understanding is uncertain, ethics being uncertain is not a special crisis. It is the general condition.

The uncomfortable truth
Uncertainty is crippling. If something is both good and bad, what do you do? The paralysis is honest. It is the correct response to not knowing.

Two Principles for Acting Under Uncertainty

Not a normative theory. Phenomenological principles for how to act when you cannot know.

Knowing You Don't Know
All assumptions fail. Constantly inspect yours. The more you learn, the more you realize you do not know.

Sounds like: paralysis
Bias to Action
When you have an idea of what is right, go do it. Learn from the feedback. You will not figure it out by thinking alone.

Sounds like: recklessness
These seem contradictory. They are not. They form a cycle.

Knowing You Don't Know

This is not just a saying. It is a practice.

Every scientific model relies on assumptions: smoothness, observability, stationarity, linearity.

Every assumption eventually fails. This is what you learned in this course.

The awareness that your knowledge rests on breakable assumptions is uncomfortable. But it is the precondition for growth.

From this course
  • Linear regression assumes linearity
  • SINDy assumes sparsity
  • PINNs assume you know the PDE
  • Neural networks assume i.i.d. data

Each is powerful within its assumptions. Each breaks outside them.

Bias to Action

The counterweight to epistemic paralysis.

You will not discover whether something works by thinking about it. You have to try it, fail, and revise.

This is the train/test loop applied to life:

1. Form a hypothesis
2. Act on it
3. Observe the gap
4. Revise the hypothesis
5. Repeat
One hour of your own thinking takes you far. Then go ask. Ask another human, ask a machine. Then try it. The feedback from trying teaches you what thinking alone never will.

The Cycle

Humility and action are not in conflict. They are phases of the same loop.

The cycle never terminates. Each pass refines your judgment. After enough passes, you develop taste.

Holding Contradictions

Radical epistemic humility plus aggressive action.

These are not in conflict. They are the same cycle, viewed from two angles.

Knowing you don't know → fuels curiosity

Curiosity → drives action

Action → generates feedback

Feedback → refines knowledge

Knowledge → reveals deeper ignorance

What Education Gives You

The traditional model: a toolbox.

For each problem type, there is a skill.

Mechanical engineering problems → statics, dynamics, thermo

Data problems → regression, classification, clustering

Optimization problems → gradient descent, linear programming

The toolbox model
Problem → Find matching skill → Apply skill → Solution

This is valid. The bigger your toolbox, the more problems you can handle.
But it is incomplete.

Why the Toolbox Is Not Enough

In a world with language models, anyone can look up any skill.

You do not need an engineering degree to have access to a toolbox. You need a search engine.

"This is my problem. What tool do I need? What is the solution?" The language model answers that.

If skill lookup is commoditized, what is left?

What is not commoditized
  • Knowing which problem to solve
  • Framing it well
  • Judging whether a solution is good enough
  • Knowing when to stop and when to push

This is taste.

Problem Definition as Meta-Skill

Real problems are not given to you pre-defined. You have to define them.

There are infinitely many ways to define a problem.

You tend to define problems using the tools you already have. "I know ML, so this is an ML problem."

But maybe it is a social problem. Or a design problem. Or a policy problem.

How to get better at it
  • Look at problem definitions outside your area
  • Be aware of what you don't know
  • Talk to people in other fields
  • Notice when your framing feels forced

Taste Through Iteration

Taste is not taught. It emerges from repeated cycles of failure and revision.

1. Define the problem (your best guess)
2. Find and apply a skill
3. It does not fully work. The problem was not quite right.
4. Redefine the problem. Try again.
5. Repeat. Each pass refines your internal model.
After enough iterations, you skip the loop. You see the problem and intuit the right framing. That is taste.

Failure Is the Curriculum

If you have never failed, you cannot tolerate the risk required for genuinely hard problems.

Startups fail ~90% of the time. Successful founders have usually failed before. They know what failure feels like, so they can tolerate the risk.

If your education has been frictionless, you have never developed tolerance for failure. You will avoid hard problems.

The homework paradox
If you take a homework assignment and plug it into a language model, you get the answer. But you skip the failure. You miss the opportunity to develop taste.

The struggle is not a bug in education. It is the entire point.

Life Is Not a Staircase

The staircase model
Grade 1 → 2 → 3 → 4
Accumulate credentials
Linear ascent
Each step is "progress"

Comfortable. Predictable. Incomplete.
The explorer model
Climb a mountain. Descend.
Travel somewhere new. Start over.
Become expert. Pivot. Begin again.
Nonlinear. Surprising. Real.

Uncomfortable. Unpredictable. True.
Your knowledge of science, of ethics, of building things: they are all connected. The same uncertainty governs them. The same iterative loop trains them. The same humility is required.

Takeaways

Think Probabilistically
Tools have distributions of consequences, not single purposes. Map the tree. Shift the mean. Narrow the spread.
Hold Contradictions
Humility + action. Good + bad in the same tool. Uncertainty is the general condition, not a special case.
Develop Taste
Define problems, fail, redefine. The iteration is the education. Struggle is not the obstacle. It is the path.

"Reality requires you to have taste, requires you to fail, and through the effort of building things and failing, you become good at your art."