Ethics, Uncertainty, and the Art of Engineering
ML for Science and Engineering - Lecture 29
Tools, responsibility, and how to act when you cannot know what is right
Joseph Bakarji
You Are a Tool
Engineers build tools. That is the job.
You learn to build small tools, then bigger tools.
When you build tools for a company, in a structural sense, you are a tool. The company uses you to build its tools.
The concept of using something and being used generalizes across humans and machines.
Key insight: This is not cynical. It is structural. Understanding where you sit in the chain is the first step toward acting with intention.
Means to Ends
Tools are means to ends. Trace any tool far enough and you arrive at a small set of human ends.
Hammer
→
Nail
→
Join wood
→
House
→
Shelter
→
Survival
→
Well-being
There are many tools, but the ends are few: survival, comfort, happiness, meaning. Most tools exist not merely for survival, but for well-being.
Nested Scales
You are building a screw in a very large machine.
Biology
A cell in your hand does not know why the hand is moving. A brain cell is not aware of the intentions of the whole brain.
Engineering
You build a component. It fits in a subsystem. The subsystem fits in a product. The product fits in a market. The market fits in society.
As technology scales globally, the distance between the tool you build and the purpose it serves keeps increasing. We are less and less sure what the tools we build are for.
AI Makes This Worse
AI systems are not only part of larger opaque systems, they are themselves unpredictable.
Traditional tool: You know the input-output relationship. The wrench turns the bolt.
AI tool: You release a neural network. People use it in ways you did not anticipate. The consequences branch unpredictably.
Double opacity: You cannot see the larger system your tool fits into, AND you cannot predict what your tool itself will do.
The Intelligence-Predictability Tradeoff
Unpredictability is not a defect. It is what makes intelligence interesting.
A perfectly predictable tool is not creative.
If you hire someone to be creative, you must tolerate unpredictability. You do not know what ideas they will produce.
The same holds for AI. If you want it to be useful in genuinely new situations, it will surprise you.
The tradeoff
Predictable ↔ Creative
Controllable ↔ Intelligent
Safe ↔ Useful
Deterministic but Unpredictable
You already know this from this course.
Chaotic systems are deterministic. Given the same initial conditions, they produce the same trajectory. But tiny perturbations lead to vastly different outcomes.
No matter how many rules you impose on a high-dimensional system, the dimensionality defeats regulation.
From this course: The Lorenz system has only 3 variables and 3 equations. It is fully deterministic. Yet it is unpredictable beyond a short horizon. Now imagine a system with billions of parameters interacting with millions of users.
The Consequence Tree
One tool, many outcomes. Click nodes to explore branches.
Click any node to expand/collapse. Color indicates value: positive to negative.
The Distribution of Outcomes
Model consequences as a probability distribution over a value spectrum.
You want to push the mean right (more good outcomes) and narrow the spread (fewer extreme bad outcomes).
We Already Manage Dangerous Tools
Every powerful technology comes with guidance. AI should be no different.
Chemicals
"Do not drink."
Warning labels, MSDS sheets, regulated concentrations.
Medicine
Dosage instructions, contraindications, clinical trials before release.
AI
???
The industry has not converged on safety standards yet.
The gap is not that AI is fundamentally different. The gap is that we have not yet built the equivalent of clinical trials, warning labels, and dosage instructions for AI systems.
Human-Centered Testing
A practical approach to managing unpredictable tools.
1. Prototype — Build a small version of the tool
2. Observe — Give it to a small group, watch how they actually use it
3. Document — Record failure modes, unintended uses, edge cases
4. Guide — Write instructions: proper use, known risks, contraindications
5. Release — Ship with the manual, not without it
Check Your Understanding
Q: A self-driving car algorithm reduces average traffic fatalities by 40%, but introduces a new failure mode that kills 1 in 100,000 pedestrians in a way that never occurred with human drivers. A strict utilitarian would argue:
A) Deploy it. The net reduction in deaths is positive.
B) Do not deploy. Introducing a new type of harm is always wrong.
C) Deploy, but only after eliminating the new failure mode entirely.
D) It depends on whether the 1 in 100,000 consented to the risk.
Answer: A — Strict utilitarianism maximizes aggregate well-being: fewer total deaths = more total good, regardless of whether the specific harm is novel. Options B and D invoke deontological or rights-based reasoning. Option C sets an unrealistic threshold. This is precisely where utilitarianism feels unsatisfying: the families of the new victims would not be consoled by statistics.
Three Levels of Ethical Thinking
The type of question you ask determines which level you are working at.
Applied Ethics — Is this specific thing good or bad?
"Should civilians own guns?" "Should we deploy this algorithm?"
Normative Ethics — What theory should govern our judgments?
"Maximize happiness" (utilitarianism), "Follow universal rules" (deontology), "Cultivate virtuous character" (virtue ethics)
Meta-Ethics — Does good and bad even exist? On what basis?
"Is morality objective, subjective, or a social construction? Does it need grounding?"
Just as tools are nested inside bigger tools, ethical reasoning is nested inside deeper ethical reasoning. You will not find bedrock.
Applied Ethics: The Gun Debate
A concrete example of applied ethics with no clean resolution.
For
- Right to self-defense
- Protection against government overreach
- Rural areas: help is far away
- Constitutional tradition
Against
- Any civilian can kill any other
- Mass shooting statistics
- Escalation dynamics
- Children's access
You can argue endlessly at this level. Applied ethics without a framework is just debate. To make progress, you need to step up one level: what theory of good are you using?
Normative Ethics: Theories of Good
Each theory gives you a lens. Each lens distorts something.
| Theory |
Core Principle |
Limitation |
| Utilitarianism |
Maximize collective happiness |
You cannot sum happiness; ignores distribution |
| Deontology |
Follow universal moral rules |
Rules conflict; rigid in edge cases |
| Virtue Ethics |
Cultivate virtuous character traits |
Who defines virtue? Culture-dependent |
| Divine Command |
Good = what the creator commands |
Requires shared religious framework |
A theory is a model of ethics. Like scientific models, ethical theories rely on assumptions and eventually fail in some domain.
Meta-Ethics: Does Good Exist?
You have intuition that 1 + 1 = 2.
You have intuition that some things are good and some are bad.
Neither has objective grounding beyond the feeling.
Some positions:
- Moral realism: Good and bad are objective features of reality
- Subjectivism: "Good" means "I like it"
- Constructivism: Morality is a social agreement, not a discovery
- Nihilism: The value system has no grounding at all
Uncertainty All the Way Down
Here is the deeper point. It is not just ethics that is uncertain.
Your entire understanding of the world is subjective. Every model, every differential equation, every scientific framework happens inside your head.
Perhaps it is collective subjectivity. But subjective nonetheless.
If scientific understanding is uncertain, ethics being uncertain is not a special crisis. It is the general condition.
The uncomfortable truth:
Uncertainty is crippling. If something is both good and bad, what do you do? The paralysis is honest. It is the correct response to not knowing.
Check Your Understanding
Q: Which of the following best describes philosophical nihilism about values?
A) Nothing matters, so you should do whatever you want
B) Values have no objective grounding, though we may still act on subjective ones
C) Only scientific facts are real; everything else is opinion
D) Morality is determined by whoever holds the most power
Answer: B — Nihilism claims there is no objective basis for moral values. This does not mean "nothing matters" (a common caricature) or that you cannot act on preferences. It means the grounding people assume for their values does not hold. You can still choose to act well; you just cannot claim the universe mandates it.
Two Principles for Acting Under Uncertainty
Not a normative theory. Phenomenological principles for how to act when you cannot know.
Knowing You Don't Know
All assumptions fail. Constantly inspect yours. The more you learn, the more you realize you do not know.
Sounds like: paralysis
Bias to Action
When you have an idea of what is right, go do it. Learn from the feedback. You will not figure it out by thinking alone.
Sounds like: recklessness
These seem contradictory. They are not. They form a cycle.
Knowing You Don't Know
This is not just a saying. It is a practice.
Every scientific model relies on assumptions: smoothness, observability, stationarity, linearity.
Every assumption eventually fails. This is what you learned in this course.
The awareness that your knowledge rests on breakable assumptions is uncomfortable. But it is the precondition for growth.
From this course:
- Linear regression assumes linearity
- SINDy assumes sparsity
- PINNs assume you know the PDE
- Neural networks assume i.i.d. data
Each is powerful within its assumptions. Each breaks outside them.
Bias to Action
The counterweight to epistemic paralysis.
You will not discover whether something works by thinking about it. You have to try it, fail, and revise.
This is the train/test loop applied to life:
1. Form a hypothesis
2. Act on it
3. Observe the gap
4. Revise the hypothesis
5. Repeat
One hour of your own thinking takes you far. Then go ask. Ask another human, ask a machine. Then try it. The feedback from trying teaches you what thinking alone never will.
The Cycle
Humility and action are not in conflict. They are phases of the same loop.
The cycle never terminates. Each pass refines your judgment. After enough passes, you develop taste.
Holding Contradictions
Radical epistemic humility plus aggressive action.
These are not in conflict. They are the same cycle, viewed from two angles.
Knowing you don't know → fuels curiosity
Curiosity → drives action
Action → generates feedback
Feedback → refines knowledge
Knowledge → reveals deeper ignorance
What Education Gives You
The traditional model: a toolbox.
For each problem type, there is a skill.
Mechanical engineering problems → statics, dynamics, thermo
Data problems → regression, classification, clustering
Optimization problems → gradient descent, linear programming
The toolbox model:
Problem → Find matching skill → Apply skill → Solution
This is valid. The bigger your toolbox, the more problems you can handle.
But it is incomplete.
Why the Toolbox Is Not Enough
In a world with language models, anyone can look up any skill.
You do not need an engineering degree to have access to a toolbox. You need a search engine.
"This is my problem. What tool do I need? What is the solution?" The language model answers that.
If skill lookup is commoditized, what is left?
What is not commoditized:
- Knowing which problem to solve
- Framing it well
- Judging whether a solution is good enough
- Knowing when to stop and when to push
This is
taste.
Problem Definition as Meta-Skill
Real problems are not given to you pre-defined. You have to define them.
There are infinitely many ways to define a problem.
You tend to define problems using the tools you already have. "I know ML, so this is an ML problem."
But maybe it is a social problem. Or a design problem. Or a policy problem.
How to get better at it:
- Look at problem definitions outside your area
- Be aware of what you don't know
- Talk to people in other fields
- Notice when your framing feels forced
Taste Through Iteration
Taste is not taught. It emerges from repeated cycles of failure and revision.
1. Define the problem (your best guess)
2. Find and apply a skill
3. It does not fully work. The problem was not quite right.
4. Redefine the problem. Try again.
5. Repeat. Each pass refines your internal model.
After enough iterations, you skip the loop. You see the problem and intuit the right framing. That is taste.
Failure Is the Curriculum
If you have never failed, you cannot tolerate the risk required for genuinely hard problems.
Startups fail ~90% of the time. Successful founders have usually failed before. They know what failure feels like, so they can tolerate the risk.
If your education has been frictionless, you have never developed tolerance for failure. You will avoid hard problems.
The homework paradox:
If you take a homework assignment and plug it into a language model, you get the answer. But you skip the failure. You miss the opportunity to develop taste.
The struggle is not a bug in education. It is the entire point.
Life Is Not a Staircase
The staircase model
Grade 1 → 2 → 3 → 4
Accumulate credentials
Linear ascent
Each step is "progress"
Comfortable. Predictable. Incomplete.
The explorer model
Climb a mountain. Descend.
Travel somewhere new. Start over.
Become expert. Pivot. Begin again.
Nonlinear. Surprising. Real.
Uncomfortable. Unpredictable. True.
Your knowledge of science, of ethics, of building things: they are all connected. The same uncertainty governs them. The same iterative loop trains them. The same humility is required.
Takeaways
Think Probabilistically
Tools have distributions of consequences, not single purposes. Map the tree. Shift the mean. Narrow the spread.
Hold Contradictions
Humility + action. Good + bad in the same tool. Uncertainty is the general condition, not a special case.
Develop Taste
Define problems, fail, redefine. The iteration is the education. Struggle is not the obstacle. It is the path.
"Reality requires you to have taste, requires you to fail, and through the effort of building things and failing, you become good at your art."