1 The language of mathematics and prompting

1.1 Prompting and learning

Generative AI has come a very long way since the introduction of ChatGPT on November 30, 2022, and it keeps evolving with explosive speed. Frontier large language models (LLM) do college level mathematics (and computer science) superbly. So much so, that the use of generative AI, in any form, is in no way allowed during the written exam in this course.
(1.1) REMARK (The LLM landscape, updated August, 2026). The past year has seen generative AI once more developing at breakneck speed. Agents started entering the workflow in November 2025 and were matured in Claude Code, Codex, OpenClaw and the like in the beginning of 2026. Frontier models like Mythos was held back because of cybersecurity risks. Over the summer incredibly powerful models like Fable and GPT 5.6 helped solve hard open research problems in mathematics. In fact, Fable helped solve one of the most difficult math problems during the FIFA World Cup final. This was casually announced on X by Levent Alpöge in the tweet below
Make no mistake, this was a genuine really hard mathematical problem dating back to 1939, that captured the attention of very capable mathematicians.
When I write helped solve, it means that there was a human in the loop guiding the LLM. Oftentimes the LLM does a lot of the heavy lifting and also contributes central ideas.
The natural interface to LLMs is the chatbot in the browser. Through your AU university account you have access to Microsoft Copilot.
You communicate with chatbots through natural language. This process is called prompting. The more precise your prompt is, the better the response. When learning new material, you can work from prompts instructing the chatbot not to give away the answers but emphasize guidance. The exercise below is a good example of this.

Chatbot Prompt

I am a student following the course based on the attached notes. Please guide me through Exercise 1.8. Emphasize my learning and do not give me the answers but only hints. You must only use material from the attached notes in the solution. Be sure to reference what you use. Please make a serious effort to render the mathematics in your output using KaTeX so that I can read it!
A click on a chatbot link copies the prompt to the clipboard and takes you to the chatbot, where you can paste the prompt. Click on a chatbot of your choice. Then attach the pdf version imo26.pdf of the interactive notes. Submit and interact.
Try different scenarios and chatbots. Browse through the beginning of the notes and change the prompt to suit you.
Gemini has a mode called guided learning and ChatGPT has something called study mode that you may also use. These are modes that support learning in the conversation with the chatbot.
In any case, precise prompting is a very valuable skill.
In this chapter several examples of prompts will be given. In the following chapters less so. Here you are expected to prompt the chatbots on your own. Sometimes a prompt related to the context pops up as below.

Chatbot Prompt

Please solve the equation $x^2 - x + 1 = 0$. Guide me through the steps. Make sure that the underlying logic in your arguments is correct.
Come up with prompts that make a chatbot act like a mathematics tutor for you. Here is a small example that you may extend.

Chatbot Prompt

Please act like a friendly tutor and teach me about the derivatives of simple functions. Test my understanding after each concept you explain.
Try out the features guided learning in Gemini and study mode in ChatGPT on the example above.
Start your \LaTeX{} journey using the prompt below.

Chatbot Prompt

I am doing weekly exercises in mathematics at the college level. Please suggest a very simple template in LaTeX for hand in of these exercises. Also, show me how to typeset an equation in LaTeX.

There seems to be a web interface to LaTeX called Overleaf. Please tell me how to access this so that I can enter a weekly exercise.
Come up with your own prompt for a question related to software.
Below I ask for feedback from the chatbot on some dubious chunk of mathematics.

Chatbot Prompt

Please give feedback on the mathematics contained in the $\LaTeX$ below in triple quotes. Emphasize logic and precision. """
$$
x^2 = 1 \implies x = 1
$$
From this it follows that $1 + 1 = 3$. """
Insert your own mathematics in \LaTeX{} notation and ask for feedback in a prompt.

1.1.1 AI atrophy

AI atrophy is the gradual weakening of human thinking by outsourcing cognitive efforts to AI. A typical example of this phenomenon is given in the prompt below.

Chatbot Prompt

Please give me the complete solution to Exercise 3.44 in the attached pdf file. Be sure to use only mathematics from this attached file and referencing precisely the proper definitions, propositions, theorems, etc. Give your answer as the source code for perfectly formatted LaTeX.
Here every inch of the cognitive effort is outsourced. Once in a while we all crawl down this rabbit hole. Personally, I get depressed using such mindless interaction. More importantly, it is arguably the worst way of preparing for the exam in this course.
I cannot emphasize enough that approaching learning in this way is extremely ineffective. The Zen state of learning in this day and age is embodied in the quote below due to Andrej Karpathy.
You can outsource your thinking, but you cannot outsoruce your understanding.
But make no mistake. To gain understanding you also need to do some thinking. Only later when your understanding is on firm ground this outsourcing to AI makes sense.

1.2 Python as your mathematical laboratory

Computers are exceptionally fun, but be careful! Nothing really beats a clear thinking human mind. To wit, I asked WolframAlpha to solve a certain optimization problem and it came up with the answer
Prompt a chatbot with

Chatbot Prompt

What is
$$
-\sqrt{\frac{1}{2} (1 - \sqrt{2} + \sqrt{3 - 2\sqrt{2}})}
$$
and explain why the output from WolframAlpha is weird. Use prompting to make it explain the mathematics input notation.
Finally use your own mental powers (and feedback from the chatbot) to explain what the proper output should have been.
We will use the programming language Python in exploring and experimenting with mathematics. The interactive notes contain live Python cells: you edit small snippets of code directly on the page, press Run and the code executes in your browser. No installation is needed. Along with Python come powerful libraries: numpy for numerics, matplotlib for plotting and sympy for computer algebra i.e., exact symbolic computation with numbers, symbols and equations.
First adjust the prompt below according to your needs and get feedback from a chatbot.

Chatbot Prompt

I am taking a mathematics course, where I explore mathematics in the browser by running small snippets of python code using the libraries numpy, matplotlib and sympy. I have no/some/extensive prior programming experience. Give me a brief introduction to python and these three libraries. Finish your reply with a small exercise I can do. If no/some/extensive is present above in this prompt, remark this and only reply with Please select your programming experience.
Below is an example of a basic graphics command in Python. Push the Run button to evaluate.

Live Python

You can also run Python on your own computer, for example through Jupyter notebooks. For heavier computer algebra there is the full-blown system Sage, built on top of Python, which earlier editions of these notes used.
Did you notice that you can edit and enter new commands in the Python cell? Do the following problems using Python — the documentation for numpy, matplotlib and sympy is helpful, and so is asking a chatbot.
  1. Consider . Plot the graph of from to . Computing does not make sense. Do you see a way of assigning a natural value to using the graph?
  2. Find an approximate solution with four decimals to the equation .
    This is an example of an equation, that can only be solved numerically. Try first plotting the graph of from to . Then use a suitable function from scipy or sympy.
  3. Compute with decimals.

Chatbot Prompt

Give me python code using sympy to compute pi with 100 decimals. I want a one line command.
Compute the sum
What is the elegant answer? Explain!
Bonus question
Generalize your answer/method to computing the sum
for .
We need a precise setup for communicating mathematics. This involves the introduction of propositional logic, predicates and sets. Let me emphasize, that this is an introductory course and not a rigorous introduction to mathematics. As such it is an organic approach, where I hope that you will return and fill out the gaps instead of getting overwhelmed by formal details already from the beginning.
However, the underlying goal is to show that mathematical precision and proofs are similar to constructing correct computer programs.
In fact, this whole first chapter may be viewed as the beginning of a computer program, where we state the exact definitions for use in the following chapters. The plan is this: first the language of mathematics (propositional logic and sets), then its fundamental objects (numbers and their ordering), then the rules of reasoning (mathematical proofs, including induction and proof by contradiction) and finally the concept of a function — ending with a first look at neural networks, the machinery driving modern AI.

1.3 Propositional logic and predicates

A proposition is a (mathematical) statement or sentence that is true () or false (). This could be a boolean expression in a computer program, like or an everyday outburst like I am tired. Both of these can be assigned true or false in a meaningful way.
In examples we will freely use numbers you know from school, like , and occasionally a set — simply a collection of objects, where means that belongs to . Sets and numbers get their full formal treatment in the two sections following this one.
Python

Live Python

Later we will see propositions with variables in them like . These are called predicates.
Propositions can be combined into new (compound) propositions. Take for example the propositions : it rains and : it is cloudy.
Then ( and ) is a perfectly good new proposition reading it rains and it is cloudy. The same goes for (if then ), which reads if it rains then it is cloudy. The proposition (if then ) reads if it is cloudy then it rains. This proposition is (clearly) false.
We need some notation to describe these compound propositions:
The compound propositions are either true() or false () depending on and . The dependencies are displayed in the truth tables below.
The tables for the compound propositions and also are not too hard to grasp. The table for raises a few more questions. Why is true? I will not go into this at this point (see Example 1.23 ), but just point out that there are many explanations available online and, perhaps more importantly, refer you to Exercise 1.11 .
Suppose that we are presented with four cards
with a (natural) number on the front and the color blue or red on the back. In (1.1) , the first and third cards are shown with their fronts facing up and the second and fourth cards are shown with their backs facing up.
A claim (proposition) is made that if a card has an even number on the front, then it must have the color blue on the back.
Your task is to verify this for the cards above. Of course you can do this by turning all four cards, but is there a way of checking this by turning less than four cards?
What if we add the claim, that if a card has the color blue on the back, then it must have an even number on the front?
Find two propositions and so that the claim reads .
A prosecutor says to the defendant: "If you committed this crime you did not act alone". Explain why the defendant should not answer "no, that is not true" here.
Explain why Python thinks that the value 1 Thanks to Gerth Brodal for pointing this out to me of

Live Python

is False! Notice that you are dividing one by zero in the last "integer" above.

1.3.1 Propositional logic as a formal language

The entities and above may have real world interpretations like it rains or it is cloudy, but we will view them as variables that can be assigned the values true or false. Independent of this assignment we define a proposition as follows.
A proposition in the variables is an expression involving the symbols that can be generated using the rules below
  1. The variables are (atomic) propositions.
  2. If is a proposition, then is a proposition.
  3. If and are propositions, then , and are propositions.
The expression is a proposition. Let us see how it is generated using the rules in Definition 1.14 .
  1. First, is a proposition using (ⅰ) .
  2. Then is a proposition by using (ⅱ) with , since we know by (1) that is a proposition.
  3. Since is a proposition by (ⅰ) , it follows that is a proposition using (ⅲ) with and , since we know by (2) that is a proposition.
  4. Finally, since is a proposition by (ⅰ) it follows by (ⅲ) with and that
    is a proposition, since we know by (3) that is a proposition.
According to the generating rules in Definition 1.14 , which of the following expressions are propositions in the variables .

1.3.2 Truth tables and equivalent propositions

Given a proposition, it makes sense to substitute values ( or ) for the variables and evaluate it using the rules in Definition 1.10 , since the parentheses leave no ambiguity as to how the evaluation must take place. For a given proposition in the variables , there are ways of assignments to the set of variables. Each of these assignments results in the value true or false after evaluation. This is conveniently recorded in the truth table of the proposition as illustrated in the example below, where so that the truth table has rows.
The truth tables corresponding to the propositions and are given below.
For example, if and , then
and
The two propositions in Example 1.17 have identical truth tables. In general, if two propositions and have identical truth tables we call them equivalent and write
In Example 1.17 we saw that
The definition below is very important to keep in mind.
The notation is used frequently. It means that both and are true i.e.,

1.3.3 Computing truth tables in Python

Python may be used to compute truth tables for propositions using the logic module in sympy. Below is an example. Be sure to check how to enter and .

Live Python

Construct by hand the truth table for the proposition .
Convince yourself either using Python or by writing out truth tables that

1.3.4 Variables, predicates and quantification

(1.21) DEFINITION (predicate). A predicate is a proposition depending on one or more variables.
Variables are fundamental in computer programs. In the predicate , appears as a variable. Depending on what you substitute for , the resulting proposition could be true or false or even meaningless. As an example, the latter case appears if is replaced by the character 'a'. This is what computer scientists call a type error. You cannot compare a character with a digit.
If , then is true, whereas is false.
is a predicate in two variables and . Here is true, whereas is false.

For every and there exists

Suppose that we have a predicate , such that is a proposition for , where is some set. Then we define the proposition
to be true if there exists , such that is true. We let
be the proposition defined by
In other words, (1.3) says that is true for every , since there does not exist making false. Let me be absolutely clear. To show that is false, it is enough to find just a single so that is false.
Here is a statement about real numbers
This statement reads: no matter which real number you pick, if , then . We definitely want this to be true. Being true means that (1.4) must hold for all numbers , also , which reads
The above statement is an example of a false implies true statement, which we want to be true.
In general terms, in proving the statement that holds for every in some set , we are really only interested in for which is true, since is our assumption. We still need to be true for for which is false. This is assured by the truth table for , since and are both true.
The following is an excerpt from the infamous Beredskabsprøve Datalogi.
Which of the following are true?

1.4 Sets

Propositions are important, but are confined by the binary values of true and false. We would like to work mathematically with objects like integers, floating point numbers, neural networks, computer programs and so on.

1.4.1 Objects and equality

One of the cornerstones of modern mathematics is deciding when two objects are the same i.e., given two objects and , deciding whether the proposition is true or false. Oftentimes an algorithm for evaluating is needed.
You may laugh here, but this is not always that easy. Even though objects appear different they are the same as in, for example the propositions
The first proposition above is an identity of fractions (rational numbers). The second is an identity, which calls for knowledge of the sine function and real numbers. Each of these identities calls for some rather advanced mathematics. The first proposition is true in a very precise way, since .

Live Python

Use the Python cell above to reason about equality in the quiz below. In each case describe the objects i.e., are they numbers, symbols, etc.? Also, please check your computations by hand with the old fashioned paper and pencil, especially .
Click on the right equalities below.
You know that . Use Python to find similar identities for and .
Go back and look at (the beginning of) Exercise 1.25 .
For two objects and we will use the notation for the proposition .
So far we have used sets informally as collections of distinct objects or elements. We introduce set theory properly here. A set is also an object as described in section 1.4.1 and it makes sense to ask when two sets are equal.
Two sets and are equal i.e., if they contain the same elements.
An example of a set could be the set of natural numbers between and . Notice again that we use the symbol "" to start the listing of elements in a set and the symbol "" to denote the end of the listing. Notice also that (by our definition of equality between sets), the order of the elements in the listing does not matter i.e.,
We are also not allowing duplicates like for example in the listing (such a thing is called a multiset).
An example of a set not involving numbers could be the set of letters
used in this sentence. The number of elements in a set is called the cardinality of the set. We will denote it by .
To convince someone beyond a doubt (we will talk about this formally later in this chapter) that two sets and are equal, one needs to argue that if is an element of , then is an element of and the other way round, if is an element of , then is an element of . If this is true, then and must contain the same elements.
Give a precise reason as to why the two sets and are not equal. Is it possible for a set with elements to be equal to a set with elements?
Sets may be explored using python. This is illustrated in the snippet below.

Live Python

Come up with three lines of python code that verifies . Try it out.

The empty set

There is a unique set containing no or zero elements. This set is called the empty set and is denoted i.e.,

Live Python

For some reason (perhaps a good one) python does not accept as input for the empty set. Why is this? Evaluate the python snippet below and explain.

Live Python

1.4.2 The symbols and

The symbol is ubiquitous in set theory (and mathematics). If is a set, then
is a proposition. It is true if is an element of or belongs to . The notation
is defined by the proposition . Also, as a bit of short hand notation, we will write
Belongs to () is straightforward in python.

Live Python

, but
. This exercise actually has
possible correct solutions if is in the second empty box and in the fourth empty box.

1.4.3 Subsets

If and are sets, then 2 At times, the symbol is used instead of . In our context these two symbols mean the same. However, the notation means that and . For example, and . means that every element of is an element of . So is a placeholder for the proposition
In this case we say that is a subset of . We also use the notation to indicate that and . In this case we say that is a strict subset of .
List the subsets of . How many are there?
It turns out that the empty set is a subset of any set.

Live Python

Explain why this is so using the definition of .

Chatbot Prompt

Explain precisely in terms of propositions and logic why the empty set is a subset of any given set.
Below Python will list all subsets of the set . Before pressing the Run button, try to write them down on your own.

Live Python

List all the subsets of a set with five elements. In general, how many subsets does a set with elements have?
The set
is not a subset of
, simply because
does not belong to . This exercise actually has
possible correct solutions.
The empty set has
elements. A set with
elements has
subsets. In general a set with elements has
subsets.

1.4.4 Set-builder notation

If is a set and a predicate for , then we build the subset
of such that is true.
Suppose that and
Then
Python
This notation has found its way to several programming languages like list comprehension in python.

Live Python

Suppose that are predicates with a variable taking values in , then we often use the notation (using instead of )
List the elements in the following subsets.
Consider the predicate
Write down the elements in
Is
an infinite set?
Explore the fascinating world of prime numbers and learn about twin primes.
You have previously encountered systems of linear equations like
The solutions to (1.7) can be identified with a subset of . Define this subset precisely i.e., write the subset as
where is a predicate in the variables .
Suppose that and
Then write down precisely what is i.e., find suitable predicates in the variable , such that and
Consider the subset of pictured in the drawing below
Express as
where are predicates in the variables .
Hint
A predicate in the variables could be something like
Express as
where
and and are suitable predicates in the variables .

1.4.5 Intersections, unions and the symbols and

Suppose that we have two sets and . Then the intersection is the set consisting of the elements in both and i.e.,
This is illustrated in the socalled Venn diagram below.
The union is the set consisting of the elements in or i.e.,
This is illustrated in the Venn diagram below.
Lastly, the difference (between and ) consists of the elements in not contained in i.e.,
This is illustrated in the Venn diagram below.
Python
You should experiment using the python window below to get a feeling for these three operations.

Live Python

Suppose that , and . What is ?
Let , and . Verify by hand (no computer) that
  1. .
  2. .
  3. .
  4. .
Given two sets and , is it true that and ?
What about ?
Suppose that and are two finite sets. Is it true that
What about
Seriously, both formulas are wrong. Can you come up with the correct version of the formula for ?
Use your correct formula to find a formula for
viewing as the first set and as the second set. Here you need the formula
Why is this formula true? Finally, explain why
Hint
You may find it useful to notice that two sets are equal i.e, if and only if
Also,
There is one more operation called the symmetric difference between two sets and . It is denoted . Experiment in the python window below to find out exactly what it does. Is it true that ?

Live Python

Here is another excerpt from the Beredskabsprøve Datalogi.
Let and denote sets. Which of the following are true?
For some sets and we can have

1.4.6 Pairs, triples and tuples

Oftentimes we want to consider more than one variable as input to a predicate. It is convenient to group the variables into one object consisting of the variables. This is done using tuples.
Given two sets and , we combine two elements and into a pair , which is an element of the Cartesian product
of and . This is a new set built from and .
If and , then
Consider two pairs and each from in . When is ?
Python
The Cartesian product can be computed in python as shown below.

Live Python

There is no need to restrict ourselves to pairs. We might as well consider triples i.e., the set of all , where , and are sets, or for that matter general tuples
of any length , where . Based on the above example with tuples we have,
You may check this using the python snippet below.

Live Python

For a given set and we define the -fold cartesian product of as
Formally is the set of pairs , where . Is there a natural way of drawing elements in ?
Let and be two sets. Is ?
Let be any set. What is ?
Let and be four sets. Is
Is
See Exercise 1.53 .
Use python to solve Exercise 1.52 by playing with (and extending) the code below.

Live Python

1.5 Numbers and their ordering

Our fundamental mathematical objects in this course are numbers. With logic and sets in our toolbox, we can introduce them precisely.

1.5.1 The natural numbers and the integers

The set of natural numbers is
The set of integers is
These are infinite sets, since they contain infinitely many elements as indicated by the dots
It makes sense to add and multiply two integers and . We will denote their addition or sum as and their multiplication or product as . A fundamental fact is that addition and multiplication are commutative i.e., and .
Please notice right away that expressions like and are complete nonsense for three integers and . We only know how to add and multiply two integers, not three. A wonderful fact comes to our rescue:
You get the same result no matter if you start adding (multiplying) and or and and then adding (multiplying) or . So we may write and as a placeholder for one of the two ways of computing this expression in (1.11) .
As you can see we use the symbol for addition, but no symbol for multiplication. This is the convention in (clean) mathematics as opposed to the coming from computer algebra (except perhaps in Mathematica or Wolfram language, where space is allowed for multiplication). However, when one of the factors is an actual number, we will use , so that for example times is written as and times is written .

1.5.2 The rational numbers

The natural numbers and the integers are well defined by their representations in (1.9) and (1.10) .
A rational number consists of a numerator and a denominator .
If and . Then and are considered equal i.e.,
if and only if .
So there are many different ways of representing a rational number, such as
Here
In fact, a fraction stays the same when its numerator and denominator are multiplied by the same natural number.
I will assume that you know how to add and multiply fractions, and that you do not make mistakes like
You can see now that this has to be wrong! In fact, according to Definition 1.54 for this addition to make sense, it must satisfy
This is nonsense, since is not equal to according to Definition 1.54 as .
In fact, if you temporarily forgot how to add fractions, you can use the wisdom in Definition 1.54 . You can replace by and by and then add the numerators as in
The computation above says that it is straightforward to add pizza slices of the same size (one sixth), but that you need to think a bit when adding one half pizza slice and two pizza slices of size one third. In general,
Click on the right equalities below. Do not use Python (or any computer)!

1.5.3 The real numbers

A real number is defined by
where and is an infinite sequence of integers (digits) in .
Informally (1.13) represents the real number
given by an infinite set of digits as opposed to a rational number, which is given finitely by an integer (numerator) and a natural number (denominator). The format in (1.13) is the usual (floating point) output from your pocket calculator or computer algebra system. For example,
The decimal expansion of above looks chaotic, but it eventually repeats itself after digits. In fact the decimal expansion of every rational number is periodic i.e., it repeats itself from a certain point.
In the definition (1.13) of a real number, we are forced to make identifications as in the definition of a rational number. I will not go into details here, but just notice that the identification
is forced upon us: if
Then
Therefore we must have .
A real number that is not rational is called irrational. There are many more irrational numbers than rational ones. Famous ones are and . The irrational number is a root in the polynomial with integer coefficients (it is an algebraic number). The numbers and are not even algebraic (they are transcendental).
In the display of a calculator you see the number . How is this number represented in the representation given in (1.13) ?

1.5.4 Arithmetic rules for numbers

Suppose that is one of the sets or . Then for numbers all in we have
If is one of or , then and for every ,
  1. for some number .
Finally if is or and is not ,then
  1. for some number .
The number above is called the inverse of and is usually denoted . The number above is called the negative of and is usually denoted . We will also use the symbol (minus) defined as an operation on two numbers as
Argue precisely that for using Proposition 1.59 .
The long list of arithmetic rules above may seem complicated at first, but they are just a formal version of what you already know, such as for example, , if and for . Beware however, that the precision in Proposition 1.59 is necessary when programming a computer.
The rules (ⅳ) and (ⅴ) are called the associative laws for addition and multiplication respectively. The rule (ⅵ) is called the distributive law. It connects multiplication with addition.
We know that zero times any number is zero. Deduce this from the rules in Proposition 1.59 starting with .
Verify that (ⅵ) is true for some specific non-zero numbers. Also convince yourself that WolframAlpha actually accepts space (between numbers and variables) as multiplication.
Suppose that and . It seems that computing involves two multiplications and one addition. Multiplications are expensive operations on a computer. Is there a way of computing with only one multiplication and one addition?
Suppose that . Use the distributive law to show that

1.5.5 Ordering numbers

In real life, finding the optimal solution to a problem often involves comparing numbers and finding the minimal or maximal one. To do this you need to be able to compare numbers in a precise way.
Let us be a little rigorous and introduce the (usual) ordering on our numbers with addition and multiplication using almost full blown mathematical formalities. The natural order on the natural numbers is
For the numbers and it is less obvious how to define an order. Mathematical simplicity comes to the rescue here. It is enough to define what (the) positive numbers are! We want (the) positive numbers to satisfy the conditions below.
A subset of positive numbers in a set of numbers must satisfy
  1. For every one and only one of the following conditions must hold
  2. If , then and .
For a set of positive numbers in , we define
and
We will write if and if .
Notice that we only use arithmetic operations to define orders on numbers in Definition 1.65 . This is also how computers compare numbers algorithmically. Also if , putting makes all of the conditions in Definition 1.65 hold. If you are given an integer, it is , positive or negative. This is the content of (ⅰ) in Definition 1.65 . Also given two natural numbers, their product and sum are also natural numbers. This is the content of (ⅱ) in Definition 1.65 .
If , why is ?
Suppose that , where is a set of numbers and given by a subset of positive numbers as in Definition 1.65 . Show that

1.5.6 Ordering

As we saw in Remark 1.66 , the natural order on is defined by , so that if for . This completely agrees with our preconception that
To be precise, writing is nonsense, since is only defined for two integers.
How is one supposed to interpret for example? Go ahead and formulate (1.14) correctly comparing only two integers at a time. How does Python interpret ? Find out using the python cell below.

Live Python

What about ? What about ?
Assume that and that . Then drag and drop the elements from the left to the right below to explain that .
To show that , we need to show that .
, since
This means that
By assumption .
This means that
But . Therefore,
But . Therefore,

1.5.7 Ordering

We define the positive rational numbers as
One can check that satisfies the conditions in Definition 1.65 . So formally we get
For ,
We must check when
This happens precisely when the numerator or . Therefore the condition in the proposition is satisfied.
Suppose that .
  1. Is it true in general that
  2. Is it true in some cases?
  3. Suppose that . Prove that
  4. What happens to the rational number
    when grows and becomes very big?

    Live Python

Using Definition 1.71 , you can check that , since
An easy, but surprising, way of finding a rational number strictly between these two is adding their numerators and denominators:
We wil try to explain the first inequality in mathematical general terms going through a rather formal proof consisting of five steps. These steps are given in the quiz below. Your task is to drag from the left and drop them to the right in an order, so that the proof makes sense.
After that you are supposed, on your own, to write down a precise proof of the second inequality.
Order the arguments below so that they constitute a coherent explanation of the statement that if
then
we need to show that .
We are assuming that .
Since and is a consequence of , we are done if we know this is true.
For integers we know that the rule holds. Therefore
we need to show that .
However, this is a consequence of our assumption .
To show that we need to argue that .
By definition this means that .
Similarly to the quiz above, assume that
Write down a precise argument showing that
The exercise below shows that our trick for finding rational numbers in between two given rational numbers can be made into a machine for generating all positive rational numbers!
Can you see the system in the fractions in the diagram below?
Begin by putting
If you move left, keep , put equal to and compute new by adding fractions the naive way. If you move right keep the etc.
Once you see the system, extend the diagram with the next level downwards.
Try to do binary search in a natural way in the tree for recovering the classical approximation for . Here you actually need to descend to more than depth in the tree! The famous approximation is above depth .
Is every positive fraction present in this diagram if one keeps adding levels?
Suppose that
and . Then for
we have and . If is a positive fraction, such that
show that

1.5.8 Ordering

For the real numbers we define the positive numbers as
Even though we have not precisely defined addition and multiplication of the real numbers, we claim that this definition of satisfies the conditions of Definition 1.65 .
One may prove that for every , there exists , such that . This is the archimedean property of the real numbers.
Given two distinct real numbers . Prove that there exists a rational number , such that
One other crucial property is the completeness of . It says that a non-empty subset with an upper bound i.e., , always has a smallest upper bound. The rational numbers do not share this property, since for example
does not have a smallest upper bound inside .
Armed with the language of logic and sets, and equipped with numbers, we now turn to the centerpiece of mathematics: proofs.

1.6 Mathematical proofs

We now have the language (logic and sets) and the objects (numbers) in place. This section is about the rules of the game: what a mathematical proof is, and the standard techniques for constructing one.

1.6.1 Proofs and inference rules

A proof begins with an assumption and proceeds with a sequence of logical steps called inference rules leading to a conclusion .
You have been taught how to solve equations in steps leading to a solution. Each step turns out to be an inference rule and the conclusion is the solution. Let us see how for the simple equation . Formally we want to prove
The first inference rule is i.e., we are allowed to add the same number to both sides of an equality. This implies
To be very precise we now use Proposition 1.59 (ⅳ) as an inference rule i.e.,
Then we use Proposition 1.59 (ⅷ) to conclude
and then finally, we get by Proposition 1.59 (ⅶ) that
So to solve the equation , we are actually using four (!) inference rules along the way.

1.6.2 The use of implication () and bi-implication ()

As you have seen, and are applied to link propositions in a logical argument. For example,
However, for we cannot link the two propositions by , simply because is false (for ).
Prove that
for every (see Definition 1.65 with for the precise definition of ). Write out every inference rule!
You need to be very precise here. What does mean precisely if ? In Definition 1.65 you will see that it means that . From this you need to deduce
for every
Try out

Chatbot Prompt

Is
$$
x \geq 0 \iff x^2 \geq 0
$$
a true statement? Give me 3 carefully crafted exercises training me in distinguishing $\implies$ and $\iff$. Only use basic mathematics involving numbers and arithmetic operations. After each exercise stop, ask for the answer and give valueable feedback and guidance.
and go through the exercises given to you.

1.6.3 More on mathematical proofs

Most professional mathematicians rarely think about the precise definition of a proof and would probably feel uncomfortable defining a proof precisely. During many years of training they have assimilated knowledge by experience. Therefore many proofs seem born out of witchcraft containing several magical devices.
However, many proofs appearing in respected mathematical journals, submitted by respected mathematicians, have turned out to contain errors. Recent developments in automated proof systems like Coq and Lean show promise in checking proofs like for example the famous four color theorem. These automated proof systems build on dependent type theory, which we will not go into.
Check out The Natural Number Game at
https://www.ma.imperial.ac.uk/~buzzard/xena/natural_number_game/index2.html
Here you can see Lean in action with every step of a proof spelled out!
Informally a proof of a proposition , consists in arguing that an implication is true by first assuming . Usually this is done not only through one implication , but through a series of intermediate implications
where the last proposition is . If is true, this will constitute a proof that is true. Just like in (1.14) , there is an imprecision here. Can you tell what it is?
An integer is called even if it is divisible by . So the even integers are
An integer is called odd if it is not even. So the odd integers are
Consider the proposition:
where i.e., the square of an odd integer is odd. This seems true for a first selection of examples: .
What does it mean exactly for a number to be odd? This means that it is not divisible by or that there exists another integer , such that . So
Therefore we need to show that
Notice that I had to change into in the second proposition above. The two variables are not the same: is associated with and is associated with .
Let us assume that . Now we need to argue that for some . You stare at this for a while and notice that we should use the assumption in computing :
Thus, using our assumption we may conclude that if , then
where . This completes the proof.
The beauty here is that we have verified for all odd natural numbers that their square is odd. Not just a finite selection like .
In many ways a proof is like a detailed argument in a court case, except that the rules of mathematics are universal. You need the absolute truth in the court of mathematics (or science).
Consider the proposition . Prove that
Use that , where is defined in Example 1.80 .

1.6.4 Proof by contradiction

A proposition is either true or false. This seemingly obvious statement goes by the name of the law of excluded middle and dates back to the writings of Aristotle. The law of excluded middle is key in the example below, where you are left with the feeling that you have been deprived of a fair and genuine proof.
An irrational number is a (real) number that is not rational. It is a startling fact that such numbers exist, but they do! The square root of two is an example. We will prove that there exists two irrational numbers , such that is rational.
Consider the proposition given by
Either is true or false. If is true we are done putting . If not, then must be false and is irrational. But then
and we are done putting and .
So which one is it? Is
rational or irrational? This is really advanced mathematics based on the Gelfond-Schneider theorem.
The law of excluded middle can be turned into a powerful proof technique called proof by contradiction.
Suppose we wish to establish that is true. Then we turn things upside down by assuming that is false i.e., that is true. If we then by logical deduction can show that
for some proposition , which is demonstrably false, then cannot be true (since true false is false). Therefore must be false and must be true by the law of the excluded middle. This technique is used all the time!
Let us use proof by contradiction to show that the proposition ( is an irrational number) is true. Assuming that is false, we must have that is true. But is the proposition (that is a rational number)
Here , where is the proposition
since we can assume that is not a common divisor of and by Definition 1.54 . However,
The last implication above follows from Exercise 1.81 . If is even, then for some . Therefore and
so that is also even. We have proved that implies the proposition given by
But notice that is the negation of the second part of i.e., . So if were true, then would be true, since we proved that is true. But would also be false, being the negation of a part of the true proposition . This is impossible. Therefore we must have that is false and therefore that is false. But then according to the law of the excluded middle, we must have that is true.
Consider the first prime numbers
Check that
are prime numbers by using the Python cell below (factor gives the prime factorization of a natural number).

Live Python

Is it true in general that
is a prime number?
Assume that we know that every natural number must be divisible by a prime number. Prove that there are infinitely many prime numbers using proof by contradiction.
Show how the assumption that there are only finitely many prime numbers say
leads to a contradiction by using that the natural number
must be divisible by a prime number.
Use proof by contradiction to show precisely that there does not exist a smallest positive rational number.

1.6.5 Proof by induction

A precocious Gauss 3 See the article Gauss's Day of Reckoning for some history of this anecdote. proved the formula
at the age of seven displaying remarkable ingenuity for his age. Lesser mortals usually use induction to prove this formula. Gauss was asked along with his classmates to compute the sum of all natural numbers . Using his formula he quickly came up with the correct answer . His classmates had to work for the entire lesson.
Suppose that the formula in (1.16) is viewed as a proposition . To prove the formula we need to prove it for all natural numbers (you can easily see that and are true) i.e., we need to prove
An induction proof is a way of proving this statement by showing two things:
These two statements ensure that . Therefore must be true, since we assumed true from the beginning. Similarly ensures that is true and so on. In fact we have proved for every using this technique. One can prove this using proof by contradiction and that every non-empty subset of has a first element. In general if is a subset of a set with an order , then is called a first element if
A crucial rule (or axiom) is that every non-empty subset of has a first element! Notice that this is false for .
Suppose that are infinitely many propositions given by . Then
is true if
  1. is true.
  2. is true.
Suppose by contradiction that there exists , such that is false. Then the subset
is non-empty. Therefore it has a first element . Here , since is assumed to be true. So we know that is true and that is true. But the latter implication is a contradiction, since true implies false is false.
Let us see how an induction proof plays out in the above example with the statement that
Clearly is true. We need to prove , so we assume that holds i.e., that (1.17) is true. Then we may add to both sides of (1.17) to get
Here the right hand side can be rewritten as
which is exactly what we want. This is the conjectured formula for the sum of the numbers . Therefore we have proved that and the induction proof is complete.
For a real number , the extremely useful formula
holds. Let us prove this formula by induction. For this amounts to the identity
which is true since . We let denote the identity in (1.18) . We have seen that is true. The induction step consists in proving . We can prove this by adding to the right hand side in (1.18) :
Real life application
In order to pay for a house you borrow DKK at an interest of per year. You want to pay off your debt over years by paying a fixed amount each year. How much is the fixed yearly amount you need to pay?
Let us analyze the setup: suppose that the fixed yearly amount is . We will find an equation giving us in terms of and . Put .
After one year you owe
After two years you owe
After three years you owe
In general after years you owe
Since we want to be debt free after years, the yearly payment will have to satisfy
By the formula (1.18) , we get
Here can be isolated giving the formula
With an interest rate of four percent (roughly the level at the time of writing), you pay a fixed monthly amount of around 4774 DKK for borrowing one million DKK over years.

Live Python

Verify the computation (induction step) in (1.19) i.e., explain the operations used to go from the left to the right of the two equalities.
Locate the mistake in the following fake induction proof of the curious fact that for every .
Let be the proposition . Then is true.
We wish to prove that assuming that are true:
This shows that and therefore that for every .
Prove by induction that the sum of the first odd numbers is given by the formula
i.e., for we have
Prove by induction that
i.e., for , we have
Prove by induction that
i.e., for , we have
Prove using the idea of induction that
for .
The last exercise related to induction concerns the famous pigeonhole principle. The statement itself looks innocent, well almost ridiculous, but it is very powerful. Even the go-to website mathoverflow for research mathematicians has a quite nice thread about this.
Prove the following by induction on : if items are put into containers and , then at least one container must contain more than one item.

1.7 The concept of a function

A function is a crucial concept in mathematics. In Python a simple function can be programmed like

Live Python

The code above seems to take a number and returns the number plus one. This (f) is in fact a function taking as input a number and returning as output the number plus one. Notice that we do not even know which numbers we are talking about here. In mathematics we need to have a more precise notion of a function.
The above python function could more formally be denoted as with if we are dealing with the integers, but we cannot tell from the code.
Well, to be fair ...
To be completely fair, it is possible from Python 3.5 to add type annotations to functions, so that we could write

def f(n: int) -> int: return(n+1)

in the Python code to state that the function should take values in the integers and return integers.
The precise mathematical definition of a function in terms of sets is the following. A function is a subset , such that . In words it states that a function is a subset of , containing pairs having only one second coordinate for every first coordinate.
The everyday working definition of a function is more intuitive: a machine taking input from some set and giving output in some set . The uniqueness of the output is encoded in the mathematical definition of a function.
Mathematically a function takes values from a set and returns values in a set . In details, it is denoted and the value associated with is denoted . Here is called the domain of and is called the codomain of . Less, formally is called the input set and the output set for .
Please notice that a function is a very, very general concept. It is not just something that you draw as a graph on a piece of paper. Of course, you can draw a function like :
Generally, a function is given by a machine, formula or algorithm that computes for every . Nothing more, nothing less. It really has nothing to do with a graph (even though graphs can sometimes be useful for visualizing certain functions like ).
Good examples of functions can be found in the cryptographic hash functions. They are examples of complicated functions , where is infinite and finite. Here could be data like plain text files and could be a bit number. This is the setup for the widely used sha-256 cryptographic hash function. The whole point of a cryptographic hash function is that it must be humanly impossible to compute with given 4 A pair with is called a collision . In fact, sha-256 is used in the Bitcoin block chain. The precise definition of sha-256 can be found in FIPS PUB 180-4 approved by the Secretary of Commerce.
Other interesting functions output a bounded size digital footprint (checksum) of a file (like md5). This is very useful for checking data integrity of downloads over the internet. The md5 hash is a bit number.
Instead of listing or bits for the hash value one uses hexadecimal notation with digits in 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 , a, b, c, d, e, f. A pair of hexadecimal digits then represents a byte or bits. Output from sha-256 and md5 consist of and hexadecimal digits respectively. You are welcome to experiment with these two hash functions in the Python cell below.

Live Python

What is the sha-256 hash of your name? Change a few letters and recompute. Do you see any system? What about the md5 hash function? Can you find two different strings with the same md5 hash using your computer?
I have not answered the last question myself, but I am told that it is possible to find a collision for md5 using a garden variety home computer. Browsing the internet, it seems that the two strings and given in hexadecimal notation 5 This notation represents a sequence of bytes given by pairs of hexadecimal digits by

d131dd02c5e6eec4693d9a0698aff95c2fcab58712467eab4004583eb8fb7f89 
55ad340609f4b30283e488832571415a085125e8f7cdc99fd91dbdf280373c5b 
d8823e3156348f5bae6dacd436c919c6dd53e2b487da03fd02396306d248cda0 
e99f33420f577ee8ce54b67080a80d1ec69821bcb6a8839396f9652b6ff72a70

and

d131dd02c5e6eec4693d9a0698aff95c2fcab50712467eab4004583eb8fb7f89 
55ad340609f4b30283e4888325f1415a085125e8f7cdc99fd91dbd7280373c5b 
d8823e3156348f5bae6dacd436c919c6dd53e23487da03fd02396306d248cda0 
e99f33420f577ee8ce54b67080280d1ec69821bcb6a8839396f965ab6ff72a70 

give a collision for md5. Verify that and that they give the same md5 hash. If you find a collision for sha-256 you will become world famous.

Live Python

1.7.1 When are two functions the same?

Suppose that . This way of defining a function is a bit sloppy. The domain and codomain is not defined. The correct way of defining a function also includes defining its domain and codomain as in Definition 1.95 . Two functions are the same when they have the same domain and codomain and for every .
The functions and given by and are not the same! Their domains are different.

1.7.2 Notations for defining a function

If is a function and is a finite set, then you can define using a simple table. This is best illustrated using an example. Suppose that and
Then is expressed in table form as
This is a way of writing
Very often the bracket (or Tuborg in Danish) notation is used. It is similar to if-then-else statements in programming:
defines the function that outputs if the input and if . In python we may express this as

Live Python

What is and for the function defined in (1.20) . Draw the graph of . Come up with a function , where it does not make sense to draw a graph.

1.7.3 Composition of functions

Given two functions and , where , we define a new function by
This notion calls for some reflection. We have a total of four sets in this definition: and and, not to forget, the condition that . If this last condition was not satisfied it would be meaningless to apply the function to . I hope the diagram below helps the understanding.
The concept of a function is powerful and underlies functional programming in computer science: every computation can be realized as applying a composition of functions to an argument. This is exemplified in the computer language Haskell.
Suppose that
and that and are given by the tables
Compute the table for . Show that is not injective (see Definition 1.106 below). Adjust the table for so that becomes bijective.
Consider and given by
What is as a function from to in terms of ?

1.7.4 Functions from and into products

Suppose that
as in (1.8) . Then the function given by
is called the projection on the -th coordinate.
If is a function, then
where .
Suppose that and . Then
Now suppose that and that is given by
Then

1.7.5 Injective and surjective functions

We now define three very important notions related to functions.
Let be a function. Then is called
  1. injective, if for every .
  2. surjective, if for every , there exists , such that .
  3. bijective, if it is both injective and surjective.
Is a cryptographic hash-function as defined in Example 1.97 injective?
Suppose that
and that the function is defined by the table
Is injective? Is it surjective? Is it possible to adjust the table so that becomes injective? Is it possible to adjust the table so that becomes surjective?
Consider the function given by
where . Is injective? Is surjective? Suggest how to change and so that becomes bijective.
Consider the function given by
Show that is bijective.
Write down precisely how the truth table for may be expressed in terms of a function . What are the sets and in this case?

1.7.6 The inverse function

If is bijective, then we may define a function , so that for every and for every . This function is denoted .
How do we define for ? Well, since is surjective, we may find so that . Now, we simply define
We cannot have in with , since is injective. We only have one choice for in (1.21) . Therefore (1.21) really is a good and sound definition.
Let , where be given by the table
Then is given by the table
What if the definition of in Example 1.112 is changed to
Does make sense here?
What is the inverse function of given by ? What is the inverse function of , where and ?

1.7.7 The preimage

Consider a function
where and are sets. If , then the preimage of under is defined by
Definition 1.115 is short and sweet. Here is a first example of the preimage.
Consider the function , where and given by
For , as illustrated below.
What is when and ?
Consider the function given by
and let . What is true about ?

1.7.8 Neural networks

Having defined functions and composition of functions, we can deflate the term (deep) neural network, which is often shrouded in magic and mystery.
A neural network is a special case of a function
where and . Neural networks are often compositions of many intermediate functions called (hidden) layers.
Recall from Definition 1.104 that a function such as (1.22) can be written
where are functions . Check out Example 1.105 .
In a neural network the functions are viewed as neurons 6 The weights below play the role of the strengths of the synapses feeding into a neuron. . Depending on their input they either fire or do not fire a signal. Classically this is modelled by the perceptron, which is a function of the form
for fixed numbers (called weights) and a number (called the bias). If the weighted sum is above the threshold , the neuron fires (returns the value ). If not it does not fire (returns the value ). (Some texts prefer writing the condition as and call the threshold; the bias is then . We use the bias convention, which is the standard one in machine learning.)
There is no magic hiding here. A layer is a function whose coordinate functions as in (1.23) are neurons. A neural network is a composition of layers, and deep simply means that the composition consists of many layers. All the mystery of a deep neural network is contained in the choice of the numbers and inside its neurons.
Perceptrons model weighted decisions of the kind you make every day. Should you bike to the university tomorrow? Suppose you weigh the chance of rain (in percent) against the time the bike saves you, (in minutes), and decide: bike exactly when
This is the perceptron with weights , and bias . The weights say how much each input matters and in which direction — rain counts against, saved time counts for, and one saved minute outweighs four percentage points of rain. The bias makes you demanding: with nothing speaking for the bike you stay on the bus, since . A chance of rain and saved minutes? Then : the neuron fires and you bike. The art is clearly in choosing the weights and the bias. Choosing them automatically from data is called training — the subject of Section 7.9 .
Give weights and a bias for a perceptron that computes the logical and function i.e, must satisfy
Do the same for the logical or function .
Not every simple rule is within reach of a single neuron, though. The next exercise looks innocent — it is one of the most famous problems in the history of artificial intelligence.
Is it possible to find a perceptron , such that
What if you are allowed to use a neural network composed as (one hidden layer) as sketched below? Try to build one before reading on — the next exercise walks through a construction.
Hint
A perceptron fires precisely on one side of a line in the plane. Draw the four points and and try to place a line with the two points where must return strictly on one side and the two points where must return on the other side. Is that possible?
The function in Exercise 1.123 is the logical exclusive or (): it returns exactly when one, but not both, of its inputs is . It is also denoted for input .
 (A bit of history)
The perceptron was introduced by Frank Rosenblatt in 1958 and set off enormous enthusiasm — The New York Times reported it as "the embryo of an electronic computer that will be able to walk, talk, see, write, reproduce itself and be conscious of its existence". In 1969 Marvin Minsky and Seymour Papert published a famous analysis of what perceptrons cannot do, and their key example is exactly Exercise 1.123 : a single perceptron cannot compute . Enthusiasm and funding evaporated in what became known as an AI winter. The remedy — hidden layers, as in the exercise, combined with an efficient method for computing weights and biases from data (Section 7.9 ) — eventually brought neural networks back and led to the deep networks behind modern machine learning.
So a single perceptron cannot compute — and history shows how costly that observation was. The rescue is composition, the very idea this chapter has been building. Here it is in slow motion.
Consider the three perceptrons , where
and
Let . Then is a composite function of two functions and . Write down these functions.
Hint
Have a closer look at (1.23) in order to understand how functions from to are expressed. Notice that our notation is a bit inconsistent when it comes to types. For example, the function should really be denoted instead of , since it takes input from . This is remedied in the (hopefully easy to understand) python code below.

Live Python

Chatbot Prompt

Explain the python code below to me.

def p1(v):
    (x, y) = v
    if -x - y + 3/2 > 0:
        return 1
    else:
        return 0

def p2(v):
    (x, y) = v
    if x + y - 1/2 > 0:
        return 1
    else:
        return 0

def p3(v):
    (x, y) = v
    if x + y - 3/2 > 0:
        return 1
    else:
        return 0

def h(v):
    return (p1(v), p2(v))

def g(v):
    return p3(v)

def f(v):
    return g(h(v))

Compute and . Compare with Exercise 1.123 — what have you just built?
Relate the perceptrons and to the illustration below. What do you think the red and blue line illustrate? What does it mean that a dot is solid compared to hollow? What is special about points between the red and blue lines? Try to relate and to the illustration.
(Illustration courtesy of William Heyman Krill).
The point of the hidden layer
If you solved Exercise 1.125 , you saw that precisely when
i.e., on the strip between the two lines in the illustration. This is something no single perceptron can do: a perceptron can only split the plane in two along one line. With a hidden layer, the neurons and each draw a line, and the output neuron combines the two half planes ( fires exactly when both and fire). Composing layers builds complicated regions out of simple ones — that is what depth buys. In particular answers the second question of Exercise 1.123 : this little network computes , which no single perceptron can.
You can experiment below. The green region shows where . Try changing the weights and biases and rerun.

Live Python

The output of one neuron can be used as input for other neurons in a potentially extremely complicated network:
The diagram above represents a neural network, which is a function . This function is actually a composition (represented by the hidden layers , , and the output layer):
All of the nodes above, except the ones in the input layer, represent perceptrons.
A perceptron as in (1.24) is specified by numbers: the weights and the bias . These numbers are called the parameters of the neuron. How many parameters does the neural network in the diagram above have in total? Hint
Count layer by layer. Each neuron in hidden layer takes inputs and therefore has parameters, and hidden layer consists of neurons.
Modern large language models are built in essentially the same way. When you read that a language model has billion parameters, you now know exactly what is being counted: the weights and biases of its neurons. The diagrams just get (much) bigger.
Mathematically there is no reason to insist on the step function in (1.24) . The general recipe behind a neuron is: form the weighted sum and feed the result into a fixed function called the activation function. The perceptron uses the step function, which returns for positive input and otherwise. A smooth alternative is the sigmoid function, giving neurons of the form
Here is the base of the exponential function, which you may recognize from high school — we will study it properly later in the book; for now the plot below tells you everything you need. Instead of jumping from to , a sigmoid neuron glides smoothly between the two values. This matters: training a network (see below) relies on small changes of the weights and biases producing small changes of the output, and the sudden jump of the step function ruins exactly that. The sigmoid function plays the starring role in logistic regression (Section 7.8 ). Around 2011 it was discovered that an even simpler activation function, the rectified linear unit () given by
often works better in deep networks, and it is the default choice in most modern networks, including large language models.

Live Python

Consider the perceptron with weights and bias , i.e.,
What is true?
You may rightfully ask where the weights and biases of a neural network come from. They are not chosen by a clever human. They are computed by solving an optimization problem: one searches for the parameters that make the network reproduce a large collection of known examples as well as possible. This process is called training, and making sense of it is one of the main goals of this book. The story unfolds gradually over the coming chapters and culminates in Section 7.9 , where you will train a genuine neural network yourself.
(1.129) REMARK (A chatbot is a function). What does any of this have to do with the chatbot from Section 1.1 ? Everything. Strip away the chat window and a large language model is a function
where the input is the conversation so far, encoded as a vector, and the output is a list of numbers: one probability for each symbol the model could write next. The chatbot writes by composing with itself: compute the probabilities, pick a next symbol accordingly, append it to the conversation and feed the longer text back into — again and again, one symbol at a time, until the reply is finished. There is no module for "knowing things" and no module for "writing" — just a function, applied repeatedly, whose billions of weights and biases were computed by training. In Section 7.9.5 you will train a miniature version of yourself and watch it invent Danish first names.
It is remarkable that the simple building blocks of this section suffice: a classical theorem, the universal approximation theorem, says that neural networks with just one hidden layer can approximate essentially any reasonable function as accurately as desired. Making "reasonable" precise requires the notion of a continuous function, which awaits you in Definition 5.65 .

Chatbot Prompt

Explain why a single perceptron cannot compute the logical function XOR, but a neural network with one hidden layer can. I am a first year university student. I know about functions and composition of functions, but please do not use calculus.