1 The language of mathematics and prompting
1.1 Prompting and learning
(1.1) REMARK (The LLM landscape, updated August, 2026).
The past year has seen generative AI once more developing at
breakneck speed.
Agents started entering the workflow in November 2025 and were matured in Claude Code,
Codex, OpenClaw and the like in the beginning of 2026. Frontier models like Mythos
was held back because of cybersecurity risks. Over the summer incredibly powerful
models like Fable and
GPT 5.6 helped solve hard open research
problems in mathematics. In fact, Fable helped solve one of the most
difficult math problems during the FIFA World Cup final. This was casually
announced on X by Levent Alpöge in the tweet below
Make no mistake, this was a genuine really hard mathematical problem dating back to 1939, that captured
the attention of very capable mathematicians.
When I write helped solve, it means that there was a human in the loop guiding the LLM.
Oftentimes the LLM does a lot of the heavy lifting and also contributes central ideas.
Chatbot Prompt
I am a student following the course based on the attached notes. Please guide me through Exercise 1.8. Emphasize my learning and do not give me the answers but only hints. You must only use material from the attached notes in the solution. Be sure to reference what you use. Please make a serious effort to render the mathematics in your output using KaTeX so that I can read it!
Chatbot Prompt
Please solve the equation $x^2 - x + 1 = 0$. Guide me through the steps. Make sure that the underlying logic in your arguments is correct.
Come up with prompts that make a chatbot act like a mathematics tutor for you.
Here is a small example that you may extend.
Try out the features guided learning in Gemini
and study mode in ChatGPT
on the example above.
Chatbot Prompt
Please act like a friendly tutor and teach me about the derivatives of simple functions. Test my understanding after each concept you explain.
Start your \LaTeX{} journey using the prompt below.
Come up with your own prompt for a question related to software.
Chatbot Prompt
I am doing weekly exercises in mathematics at the college level. Please suggest a very simple template in LaTeX for hand in of these exercises. Also, show me how to typeset an equation in LaTeX.
There seems to be a web interface to LaTeX called Overleaf. Please tell me how to access this so that I can enter a weekly exercise.
Below I ask for feedback from the chatbot on some dubious chunk of mathematics.
Insert your own mathematics in \LaTeX{} notation and ask for feedback in a prompt.
Chatbot Prompt
Please give feedback on the mathematics contained in the $\LaTeX$ below in triple quotes. Emphasize logic and precision. """
$$
x^2 = 1 \implies x = 1
$$
From this it follows that $1 + 1 = 3$. """
1.1.1 AI atrophy
Chatbot Prompt
Please give me the complete solution to Exercise 3.44 in the attached pdf file. Be sure to use only mathematics from this attached file and referencing precisely the proper definitions, propositions, theorems, etc. Give your answer as the source code for perfectly formatted LaTeX.
You can outsource your thinking, but you cannot outsoruce your understanding.
1.2 Python as your mathematical laboratory
Prompt a chatbot with
and explain why the output from WolframAlpha is weird.
Use prompting to make it explain the mathematics input notation.
Finally use your own mental powers (and feedback from the
chatbot) to explain what the proper output should have been.
Chatbot Prompt
What is
$$
-\sqrt{\frac{1}{2} (1 - \sqrt{2} + \sqrt{3 - 2\sqrt{2}})}
$$
Chatbot Prompt
I am taking a mathematics course, where I explore mathematics in the browser by running small snippets of python code using the libraries numpy, matplotlib and sympy. I have no/some/extensive prior programming experience. Give me a brief introduction to python and these three libraries. Finish your reply with a small exercise I can do. If no/some/extensive is present above in this prompt, remark this and only reply with Please select your programming experience.
Live Python
Did you notice that you can edit and enter new commands in the Python cell?
Do the following problems using Python — the documentation for
numpy, matplotlib
and sympy is helpful, and so is
asking a chatbot.
- Consider . Plot the graph of from to . Computing does not make sense. Do you see a way of assigning a natural value to using the graph?
- Find an approximate solution with four decimals to the equation .This is an example of an equation, that can only be solved numerically. Try first plotting the graph of from to . Then use a suitable function from scipy or sympy.
- Compute with decimals.
Chatbot Prompt
Give me python code using sympy to compute pi with 100 decimals. I want a one line command.
Compute the sum
What is the elegant answer? Explain!
Bonus question
Generalize your answer/method to computing the sum
for .
1.3 Propositional logic and predicates
In examples we will freely use numbers you know from school, like ,
and occasionally a set — simply a collection of objects, where
means that belongs to . Sets and numbers get their full
formal treatment in the two sections following this one.
Live Python
Suppose that we are presented with four cards
with a (natural) number on the front and the color
blue or red on the back.
In
(1.1)
, the first and third cards are shown with their fronts facing up and
the second and fourth cards are shown with their backs facing up.
A claim (proposition) is made that if a card has an even number on the front, then it
must have the color blue on the back.
Your task is to verify this for the cards above. Of course you can
do this by turning all four cards, but is there a way of checking this
by turning less than four cards?
What if we add the claim, that if a card has the color
blue on the back, then
it must have an even number on the front?
Find two propositions and so that the claim reads
.
A prosecutor says to the defendant: "If you committed this crime you did not act alone".
Explain why the defendant should not answer "no, that is not true" here.
Explain why Python thinks that the value
1
Thanks to Gerth Brodal for pointing this out to me
of
is False! Notice that you are dividing one by zero in the last "integer" above.
Live Python
1.3.1 Propositional logic as a formal language
A proposition in the variables is an expression involving
the symbols that
can be generated using the rules below
- The variables are (atomic) propositions.
- If is a proposition, then is a proposition.
- If and are propositions, then , and are propositions.
The expression is a proposition. Let us see how it is generated
using the rules in Definition
1.14
.
- First, is a proposition using (ⅰ) .
- Then is a proposition by using (ⅱ) with , since we know by (1) that is a proposition.
- Since is a proposition by (ⅰ) , it follows that is a proposition using (ⅲ) with and , since we know by (2) that is a proposition.
- Finally, since is a proposition by (ⅰ) it follows by (ⅲ) with and that is a proposition, since we know by (3) that is a proposition.
1.3.2 Truth tables and equivalent propositions
The truth tables corresponding to the propositions and
are given below.
For example, if and , then
and
The notation is used frequently. It means that both
and are true i.e.,
1.3.3 Computing truth tables in Python
Live Python
Construct by hand the truth table for the proposition
.
Convince yourself either using Python or by writing out truth tables that
1.3.4 Variables, predicates and quantification
(1.21) DEFINITION (predicate).
A predicate is a proposition depending on one or more variables.
If , then is true, whereas is false.
is a predicate in two variables and . Here is true, whereas is false.
For every and there exists
Here is a statement about real numbers
This statement reads: no matter which real number you pick, if ,
then . We definitely want this to be true. Being true means
that
(1.4)
must hold for all numbers , also ,
which reads
The above statement is an example of a false implies true statement, which
we want to be true.
In general terms, in proving the statement that holds for every in
some set , we are really only interested in for which is true, since
is our assumption. We still need to be true for for
which is false. This is assured by the truth table for , since
and are both true.
1.4 Sets
1.4.1 Objects and equality
Live Python
You know that . Use Python to find similar identities
for and .
Go back and look at (the beginning of) Exercise
1.25
.
Two sets and are equal i.e., if they contain the same elements.
Give a precise reason as to why the two sets and are not equal.
Is it possible for a set with elements to be equal to a set with elements?
Live Python
Come up with three lines of python code that verifies . Try it out.
The empty set
Live Python
For some reason (perhaps a good one) python does not accept as input for the empty set.
Why is this? Evaluate the python snippet below and explain.
Live Python
1.4.2 The symbols and
Live Python
1.4.3 Subsets
List the subsets of . How many are there?
It turns out that the empty set is a subset of any set.
Explain why this is so using the definition of .
Live Python
Chatbot Prompt
Explain precisely in terms of propositions and logic why the empty set is a subset of any given set.
Below Python will list all subsets of the set . Before pressing
the Run button, try to write them down on your own.
List all the subsets of a set with five elements. In general, how many subsets does a set with elements have?
Live Python
1.4.4 Set-builder notation
Suppose that and
Then
This notation has found its way to several programming languages like list
comprehension in python.
Live Python
List the elements in the following subsets.
Consider the predicate
Write
down the elements in
Is
an infinite set?
Explore the fascinating world of prime numbers and learn about twin primes.
You have previously encountered systems of linear equations like
The solutions to
(1.7)
can be identified with a subset of
. Define this subset precisely i.e., write the subset as
where is a predicate in the variables .
Suppose that and
Then write down precisely what is i.e., find suitable predicates in the
variable , such that
and
Consider the subset of pictured in the drawing below
Express as
where are predicates in the variables .
Hint
Express as
where
and and are suitable predicates in the variables .
A predicate in the variables could be something like
1.4.5 Intersections, unions and the symbols and
You should experiment using the python window below to get a feeling for these three operations.
Live Python
Suppose that , and . What is
?
Let , and . Verify by hand (no computer) that
- .
- .
- .
- .
Given two sets and , is it true that
and ?
What about ?
Suppose that and are two finite sets. Is it true that
What about
Seriously, both formulas are wrong. Can you come up with the correct
version of the formula for ?
Use your correct formula to find a formula for
viewing as the first set and as the second set. Here you need
the formula
Why is this formula true? Finally, explain why
Hint
You
may find it useful to notice that two sets are equal i.e,
if and only if
Also,
There is one more operation called the symmetric difference between two sets and . It is
denoted . Experiment in the python window below to find out exactly what it does.
Is it true that ?
Live Python
For some sets and we can have
1.4.6 Pairs, triples and tuples
If and , then
Consider two pairs and each from in . When
is ?
The Cartesian product can be computed in python as shown below.
Live Python
Live Python
For a given set and we define the -fold cartesian product of as
Formally is the set of pairs , where . Is there a natural
way of drawing elements in ?
1.5 Numbers and their ordering
1.5.1 The natural numbers and the integers
The set of natural numbers is The set of integers is These are infinite sets, since they contain infinitely many elements as indicated by the dots1.5.2 The rational numbers
A rational number
consists of a numerator and a denominator .
If and . Then and
are considered equal i.e.,
if and only if .
So there are many different ways of representing a rational number, such as
Here
In fact, a fraction stays the same when its numerator and denominator are
multiplied by the same natural number.
1.5.3 The real numbers
A real number is defined by
where and is an infinite sequence of integers (digits) in .
In the display of a calculator you see the number . How is this number
represented in the representation given in
(1.13)
?
1.5.4 Arithmetic rules for numbers
Suppose that is one of the sets or . Then for numbers
all in we have
- for some number .
- for some number .
Argue precisely that for using Proposition
1.59
.
We know that zero times any number is zero. Deduce this from the rules in
Proposition
1.59
starting with .
Verify that
(ⅵ)
is true for some specific non-zero numbers. Also
convince yourself that WolframAlpha actually
accepts space (between numbers and variables) as multiplication.
Suppose that and . It seems
that computing involves two multiplications and one addition. Multiplications are
expensive operations on a computer. Is there a way of computing
with only one multiplication and one addition?
Suppose that . Use the distributive law to show that
1.5.5 Ordering numbers
A subset of positive numbers in a set of numbers must satisfy
- For every one and only one of the following conditions must hold
- If , then and .
Notice that we only use arithmetic operations to define orders on
numbers in Definition
1.65
.
This is also how computers compare numbers algorithmically. Also if ,
putting makes all of the conditions in Definition
1.65
hold. If you are given an integer, it is , positive or negative. This is the
content of
(ⅰ)
in Definition
1.65
.
Also given two natural numbers, their product and sum are also natural numbers. This is
the content of
(ⅱ)
in Definition
1.65
.
If , why is ?
Suppose that , where is a set of numbers and given
by a subset of positive numbers as in Definition
1.65
.
Show that
1.5.6 Ordering
How is one supposed to interpret for example? Go ahead and formulate
(1.14)
correctly comparing only two integers at a time.
How does Python interpret ? Find out using the python cell below.
What about ? What about ?
Live Python
To show that , we need to show that
.
, since
This means that
By assumption .
This means that
But . Therefore,
But . Therefore,
1.5.7 Ordering
For ,
We must check when
This happens precisely when the numerator or . Therefore
the condition in the proposition is satisfied.
Suppose that .
we need to show that .
We are assuming that .
Since and is a consequence of , we are
done if we know this is true.
For integers we know that the rule
holds. Therefore
we need to show that .
However, this is a consequence of our assumption .
To show that
we need to argue that .
By definition this means that .
Similarly to the quiz above, assume that
Write down a precise argument showing that
Can you see the system in the fractions in the diagram below?
Once you see the system, extend the diagram with the next level downwards.
Try to do binary search in a natural way in the tree for recovering the classical approximation
for .
Here you actually need to descend to more than depth in the tree!
The famous approximation is above depth .
Is every
positive fraction present in this diagram if one keeps adding levels?
Begin by putting
If you move left, keep , put equal to
and compute new by adding fractions
the naive way.
If you move right keep the etc.
Suppose that
and . Then for
we have and . If is
a positive fraction, such that
show that
1.5.8 Ordering
Given two distinct real numbers . Prove that
there exists a rational number , such that
1.6 Mathematical proofs
1.6.1 Proofs and inference rules
1.6.2 The use of implication () and bi-implication ()
Prove that
for every (see Definition
1.65
with for the precise definition of ).
Write out every inference rule!
You need to be very precise here. What does mean precisely if ? In
Definition
1.65
you will see that it means that . From this
you need to deduce
Try out
and go through the exercises given to you.
Chatbot Prompt
Is
$$
x \geq 0 \iff x^2 \geq 0
$$
a true statement? Give me 3 carefully crafted exercises training me in distinguishing $\implies$ and $\iff$. Only use basic mathematics involving numbers and arithmetic operations. After each exercise stop, ask for the answer and give valueable feedback and guidance.
1.6.3 More on mathematical proofs
Check out The Natural Number Game at
https://www.ma.imperial.ac.uk/~buzzard/xena/natural_number_game/index2.htmlHere you can see Lean in action with every step of a proof spelled out!
An integer is called even if it is divisible by . So the even integers are
An integer is called odd if it is not even. So the odd integers are
Consider the
proposition:
where i.e., the square of an odd
integer is odd. This seems true for a first selection
of examples: .
What does it mean
exactly for a number to be odd? This means that it is
not divisible by or that there exists another
integer , such that . So
Therefore we need to show that
Notice that I had to change into in the second proposition above.
The two variables are not the same: is associated with and
is associated with .
Let us assume that . Now we need to argue that for some . You stare at this for a while
and notice that we should use the assumption in
computing :
Thus, using our assumption we may conclude that if , then
where . This completes the proof.
1.6.4 Proof by contradiction
An irrational number is a (real) number that is not rational. It is a
startling fact that such numbers exist, but they do! The square root of two
is an example.
We will prove that there exists two irrational numbers , such that
is rational.
Consider the proposition given
by
Either is true or false. If is true we are done putting . If not, then
must be false and is irrational. But then
and we are done putting and .
So which one is it? Is
rational or irrational?
This is really advanced mathematics based on the
Gelfond-Schneider theorem.
Let us use proof by contradiction to show that the proposition ( is an irrational number) is true.
Assuming that is false, we must have that is true. But
is the proposition (that is a rational number)
Here , where is the proposition
since we can assume that is not a common divisor of and by
Definition
1.54
.
However,
The last implication above follows from Exercise
1.81
. If is even, then
for some . Therefore and
so that is also even. We have proved that implies the proposition given by
But notice that is the negation of the second part of i.e.,
.
So if were true, then would be true, since we proved that
is true. But would also be false, being the
negation of a part of the true proposition . This is impossible. Therefore
we must have that is false and therefore that is false. But then
according to the law of the excluded middle, we must have that is true.
Consider the first prime numbers
Check that
are prime numbers by using the Python cell below (factor gives
the prime factorization of a natural number).
Is it true in general that
is a prime number?
Assume that we know that every natural number must be divisible by a prime number.
Prove that there are infinitely many prime numbers using proof
by contradiction.
Live Python
Show how the
assumption that there are only finitely many prime numbers say
leads to a contradiction by using that the natural number
must be divisible by a prime number.
Use proof by contradiction to show precisely that there does not
exist a smallest positive rational number.
1.6.5 Proof by induction
Suppose that are infinitely many propositions given by . Then
is true if
- is true.
- is true.
Suppose by contradiction that there exists , such that
is false. Then the subset
is non-empty. Therefore it has a first element .
Here , since is assumed to be true. So we
know that is true and that
is true. But the latter
implication is a contradiction, since true implies
false is false.
For a real number , the extremely useful formula
holds. Let us prove this formula by induction. For this amounts to the identity
which is true since . We let denote
the identity in
(1.18)
. We have seen that is true. The induction step
consists in proving . We can prove this
by adding to the right hand side in
(1.18)
:
Real life application
In order to pay for a house you borrow DKK at an interest of
per year. You want to pay off your debt over years by
paying a fixed amount each year. How much is the fixed yearly
amount you need to pay?
Let us analyze the setup: suppose that the fixed yearly amount
is . We will find an equation giving us in terms of
and . Put .
After one year you owe
After two years you owe
After three years you owe
In general after years you owe
Since we want to be debt free after years, the yearly payment will have to satisfy
By the formula
(1.18)
, we get
Here can be isolated giving the formula
With an interest rate of four percent (roughly the level at the time of writing), you pay a fixed monthly
amount of around 4774 DKK for borrowing one million DKK over years.
Live Python
Verify the computation (induction step) in
(1.19)
i.e., explain
the operations used to go from the left to the right of the two equalities.
Locate the mistake in the following fake induction proof of the curious fact that
for every .
Let be
the proposition . Then is true.
We wish to prove that assuming that are true:
This shows that and therefore that for every
.
Prove by induction that the sum of the first odd numbers is
given by the formula
i.e., for we have
Prove by induction that
i.e., for , we have
Prove by induction that
i.e., for , we have
Prove using the idea of induction that
for .
Prove the following by induction on : if items are put into containers and
, then at least one container must contain more than one item.
1.7 The concept of a function
Live Python
To be completely fair, it is possible from Python 3.5 to add type annotations to functions, so that we could write
def f(n: int) -> int: return(n+1)
in the Python code to state that the function should take values in the integers and return integers.
Mathematically a function takes values from a set and returns values in a set . In details,
it is denoted and the value associated with is denoted .
Here is called the domain of and is called the codomain of . Less,
formally is called the input set and the output set for .
Please notice that a function is a very, very general concept. It is not just something
that you draw as a graph on a piece of paper. Of course, you can draw a function
like :
Generally, a function is given by a machine, formula or algorithm that
computes for every . Nothing more, nothing less. It really has nothing to
do with a graph (even though graphs can sometimes be useful for visualizing certain functions like ).
Good examples of functions can be found in the cryptographic hash functions. They are examples of complicated functions , where
is infinite and finite. Here could be data like plain text files and could be
a bit number. This is the setup for the widely used sha-256 cryptographic hash function.
The whole point of a cryptographic hash function is that it must be humanly impossible to
compute with given
4
A pair with is called a collision
.
In fact, sha-256 is used in the Bitcoin block chain. The precise definition of
sha-256 can be found in FIPS PUB 180-4 approved by the Secretary of Commerce.
Other interesting functions output a bounded size digital footprint (checksum) of a file (like md5). This is very useful
for checking data integrity of downloads over the internet. The md5 hash is a bit number.
Instead of listing or bits for the hash value one uses hexadecimal notation with digits
in 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 , a, b, c, d, e, f. A pair of hexadecimal digits then represents
a byte or bits. Output from sha-256 and md5 consist of and hexadecimal
digits respectively. You are welcome to experiment with these two hash functions in the
Python cell below.
Live Python
What is the sha-256 hash of your name? Change a
few letters and recompute. Do you see any system? What about the md5 hash function?
Can you find two different strings with the same md5 hash using your computer?
I have not answered the last question myself, but I am told that it is possible to find
a collision for md5 using a garden variety home computer. Browsing the internet, it
seems that the two strings and given in hexadecimal notation
5
This notation represents a sequence of bytes given by pairs of hexadecimal digits
by
d131dd02c5e6eec4693d9a0698aff95c2fcab58712467eab4004583eb8fb7f89
55ad340609f4b30283e488832571415a085125e8f7cdc99fd91dbdf280373c5b
d8823e3156348f5bae6dacd436c919c6dd53e2b487da03fd02396306d248cda0
e99f33420f577ee8ce54b67080a80d1ec69821bcb6a8839396f9652b6ff72a70
and
d131dd02c5e6eec4693d9a0698aff95c2fcab50712467eab4004583eb8fb7f89
55ad340609f4b30283e4888325f1415a085125e8f7cdc99fd91dbd7280373c5b
d8823e3156348f5bae6dacd436c919c6dd53e23487da03fd02396306d248cda0
e99f33420f577ee8ce54b67080280d1ec69821bcb6a8839396f965ab6ff72a70
give a collision for md5. Verify that and
that they give the same md5 hash. If you find a collision
for sha-256 you will
become world famous.Live Python
1.7.1 When are two functions the same?
The functions and
given by and are not the same! Their domains
are different.
1.7.2 Notations for defining a function
Live Python
What is and for the function defined in
(1.20)
. Draw
the graph of . Come up with a function , where it
does not make sense to draw a graph.
1.7.3 Composition of functions
The concept of a function is powerful and underlies functional programming in computer science: every computation can be realized as applying a composition of functions to an argument. This is exemplified in the computer language
Haskell.
Suppose that
and that and are given by the tables
Compute the table for . Show that is not
injective (see Definition
1.106
below). Adjust the table for so that becomes bijective.
Consider and given by
What is as a function from to in terms of ?
1.7.4 Functions from and into products
Suppose that
as in
(1.8)
. Then the function
given by
is called the projection on the -th coordinate.
If is a function, then
where .
Suppose that and . Then
Now suppose that and that is given by
Then
1.7.5 Injective and surjective functions
Let be a function. Then is called
- injective, if for every .
- surjective, if for every , there exists , such that .
- bijective, if it is both injective and surjective.
Is a cryptographic hash-function as defined in Example
1.97
injective?
Suppose that
and that the function is defined by the table
Is injective? Is it surjective? Is it possible to adjust the table so that
becomes injective?
Is it possible to adjust the table so that
becomes surjective?
Consider the function given by
where .
Is injective? Is surjective? Suggest how to change and so that becomes
bijective.
Consider the function given by
Show that is bijective.
Write down precisely how the truth table for may
be expressed in terms of a function . What are the sets and in this case?
1.7.6 The inverse function
Let , where be given by
the table
Then is given by the table
What if the definition of in Example
1.112
is changed to
Does make sense here?
What is the inverse function of given by ?
What is the inverse function of , where and
?
1.7.7 The preimage
Consider a
function
where and are sets. If , then the
preimage of under is defined by
Consider the function , where and
given by
For , as illustrated below.
What is when and
?
1.7.8 Neural networks
There is no magic hiding here. A layer is a function
whose coordinate functions as in
(1.23)
are neurons. A neural network is a composition of
layers, and deep simply means that the composition consists of many
layers. All the mystery of a deep neural network is contained in the
choice of the numbers and inside its neurons.
Perceptrons model weighted decisions of the kind you make every day.
Should you bike to the university tomorrow? Suppose you weigh the
chance of rain (in percent) against the time the bike saves
you, (in minutes), and decide: bike exactly when
This is the perceptron with weights
, and bias . The weights say how much
each input matters and in which direction — rain counts against,
saved time counts for, and one saved minute outweighs four percentage
points of rain. The bias makes you demanding: with nothing speaking
for the bike you stay on the bus, since . A
chance of rain and saved minutes? Then
: the neuron fires and you bike. The art is
clearly in choosing the weights and the bias. Choosing them
automatically from data is called training — the subject of
Section
7.9
.
Give weights and a bias for a perceptron that computes
the logical and function i.e, must satisfy
Do the same for the logical or function .
Is it possible to find a perceptron , such that
What if you are allowed to use a neural network composed as
(one hidden layer) as sketched below?
Try to build one before reading on — the next exercise walks
through a construction.
Hint
A perceptron fires precisely on one side of a line in the plane. Draw the four
points and and try to place a line with the
two points where must return strictly on one side and the two points
where must return on the other side. Is that possible?
The perceptron was introduced by
Frank Rosenblatt
in 1958 and set off enormous enthusiasm — The New York Times reported it as
"the embryo of an electronic computer that will be able to walk, talk, see,
write, reproduce itself and be conscious of its existence". In 1969
Marvin Minsky and
Seymour Papert
published a famous analysis of what perceptrons cannot do, and their key
example is exactly Exercise
1.123
: a single perceptron cannot compute
. Enthusiasm and funding evaporated in what became known as an
AI winter. The remedy —
hidden layers, as in the exercise, combined with an efficient method for
computing weights and biases from data (Section
7.9
) —
eventually brought neural networks back and led to the deep networks behind
modern machine learning.
Consider the three perceptrons , where
and
Let . Then is
a composite function of two functions
and . Write down these functions.
Hint
Compute
and . Compare with Exercise
1.123
— what have you just built?
Relate the perceptrons and to the illustration
below. What do you think the red and blue line illustrate? What does
it mean that a dot is solid compared to hollow? What is special
about points between the red and blue lines? Try to relate and to the illustration.
(Illustration courtesy of William Heyman Krill).
Have a closer look at
(1.23)
in order to understand how
functions from to are expressed. Notice that
our notation is a bit inconsistent when it comes to types. For example,
the function should really be
denoted instead of , since it takes input from
. This is remedied in the (hopefully easy to understand) python code below.
Live Python
Chatbot Prompt
Explain the python code below to me.
def p1(v):
(x, y) = v
if -x - y + 3/2 > 0:
return 1
else:
return 0
def p2(v):
(x, y) = v
if x + y - 1/2 > 0:
return 1
else:
return 0
def p3(v):
(x, y) = v
if x + y - 3/2 > 0:
return 1
else:
return 0
def h(v):
return (p1(v), p2(v))
def g(v):
return p3(v)
def f(v):
return g(h(v))
If you solved Exercise
1.125
, you saw that precisely when
i.e., on the strip between the two lines in the illustration. This is
something no single perceptron can do: a perceptron can only split the
plane in two along one line. With a hidden layer, the neurons
and each draw a line, and the output neuron combines the
two half planes ( fires exactly when both and fire).
Composing layers builds complicated regions out of simple ones — that
is what depth buys. In particular answers the second question of
Exercise
1.123
: this little network computes ,
which no single perceptron can.
You can experiment below. The green region shows where .
Try changing the weights and biases and rerun.
Live Python
A perceptron as in
(1.24)
is specified by
numbers: the weights and the bias . These numbers
are called the parameters of the neuron. How many parameters does the
neural network in the diagram above have in total?
Hint
Count layer by layer. Each neuron in hidden layer takes inputs and
therefore has parameters, and hidden layer consists of
neurons.
Modern large language models are built in essentially the same way. When you
read that a language model has billion parameters, you now know exactly
what is being counted: the weights and biases of its neurons. The diagrams
just get (much) bigger.
Live Python
Consider the perceptron with weights
and bias , i.e.,
What is true?
(1.129) REMARK (A chatbot is a function).
What does any of this have to do with the chatbot from Section
1.1
? Everything. Strip away the chat window and a large
language model is a function
where the input is the conversation so far, encoded as a vector, and the
output is a list of numbers: one probability for each symbol the model could
write next. The chatbot writes by composing with itself: compute
the probabilities, pick a next symbol accordingly, append it to the
conversation and feed the longer text back into — again and again, one
symbol at a time, until the reply is finished. There is no module for
"knowing things" and no module for "writing" — just a function, applied
repeatedly, whose billions of weights and biases were computed by training.
In Section
7.9.5
you will train a miniature
version of yourself and watch it invent Danish first names.
Chatbot Prompt
Explain why a single perceptron cannot compute the logical function XOR, but a neural network with one hidden layer can. I am a first year university student. I know about functions and composition of functions, but please do not use calculus.