Title: Part IA / Differential Equations
Tags: Second_Order_ODE First_Order_ODE Multivariate_Functions Differentiation Partial_Differentiation Integration Central_Forces

Q: Definition. Derivative of Function
A: The derivative of a function f(x) with respect to x is \[\frac{\mathrm{d}f}{\mathrm{d}x} = \lim_{h \to 0}\frac{f(x+h)-f(x)}{h}\]
The function is differentiable at x if the limit exists, i.e. the left-hand and right-hand limits are equal

Q: Definition. \(f(x) = o(g(x)) \text{ as } x \to x_0\)
A: \(\lim_{x \to x_0}\frac{f(x)}{g(x)} = 0\)

Q: Definition. \(f(x) = O(g(x)) \text{ as } x \to x_0\)
A: \(\frac{f(x)}{g(x)}\) is bounded as \(x \to x_0\)
or
\[(\exists \delta > 0 , M>0)|f(x)|\leq M|g(x)|, 0 < |x - x_0| < \delta\]

Q: Proposition. \(f(x_0 + h) = f(x_0) + f'(x_0)h + o(h)\)
A: We have \[f'(x_0) + \frac{o(h)}{h} = \frac{f(x_0 + h) - f(x_0)}{h}\]
by the definition of the derivative and the small o notation. The result follows

Q: Theorem. Chain rule, Product rule, Leibniz's rule
A: Chain rule. Given \(f(x) = F(g(x)), \frac{\mathrm{d}f}{\mathrm{d}x} = \frac{\mathrm{d}F}{\mathrm{d}g}\frac{\mathrm{d}g}{\mathrm{d}x}\)
Product rule. Given \(f(x) = u(x)v(x), f'(x) = u'(x)v(x) + u(x)v'(x)\)
Leibniz's Rule. Given \(f = uv, f^{(n)}(x) = \sum_{r=0}^n {n \choose r}u^{(r)}v^{(n-r)}\)

Q: Theorem. Taylor's Theorem
A: \(f(x+h) = f(x)+hf'(x) + \frac{h^2}{2!}f''(x) + \cdots + \frac{h^n}{n!}f^{(n)}(x) + E_n\)
or alternatively
\(f(x) = f(x_0) + (x-x_0)f'(x_0) + \cdots + \frac{(x-x_0)^n}{n!}f^{(n)}(x_0) + E_n\)

Q: Theorem. L'Hopitals Rule
Let \(f(x)\) and \(g(x)\) be differentiable at \(x_0\), and \(\lim_{x \to x_0}f(x) = \lim_{x \to x_0}g(x) = 0\), then\[\lim_{x \to x_0} \frac{f(x)}{g(x)} = \lim_{x \to x_0} \frac{f'(x)}{g'(x)}\]
A: Using Taylor's Theorem, \[\lim_{x \to x_0} \frac{f(x)}{g(x)} = \lim_{x \to x_0} \frac{f(x_0) + (x-x_0)f'(x_0) + o(x-x_0)}{g(x_0) + (x-x_0)g'(x_0) + o(x-x_0)} = \lim_{x \to x_0} \frac{f'(x_0) + \frac{o(x-x_0)}{x-x_0}}{g'(x_0) + \frac{o(x-x_0)}{x-x_0}} = \lim_{x \to x_0} \frac{f'(x)}{g'(x)}\]

Q: Definition. Integral
A: An integral is the limit of a sum, \[\int_a^b f(x) \mathrm{d}x = \lim_{\Delta x \to 0} \sum_{n=0}^N f(x_n)\Delta x\]
For the area from a to b, and if f is differentiable the integral is, \[\lim_{N \to \infty} \sum_{n=0}^{N-1}(f(x_n)\Delta x) + N \cdot O(\Delta x^2) = \lim_{N \to \infty} \sum_{n=0}^{N-1}(f(x_n)\Delta x) + O(\Delta x) = \int_a^b f(x) \mathrm{d}x\]
Where \[\Delta x = \frac{b-a}{N}, x_n = a+n\Delta x\]

Q: Theorem. Fundamental Theorem of Calculus
Let \(F(x) = \int_a^x f(t) \mathrm{d}t\). Then \(F'(x) = f(x)\)
A: \(\frac{\mathrm{d}}{\mathrm{d}x} F(x) = \lim_{h \to 0} \frac{1}{h} \left [\int_a^{x+h} f(t) \mathrm{d}t - \int_a^x f(t) \mathrm{d}t \right ] = \lim_{h \to 0} \frac{1}{h} \int_x^{x+h} f(t) \mathrm{d}t = \lim_{h \to 0} \frac{1}{h} [f(x)h + O(h^2)] = f(x)\)

Q: Theorem. Integration by parts
\(\int uv' \mathrm{d}x = uv - \int vu' \mathrm{d}x\)
A: From the product rule we have \((uv)' = uv' + u'v\), integrating the whole expression and rearranging gives the formula above

Q: Definition. Partial Derivative
A: \[\frac{\partial f}{\partial x} |_{y} = \lim_{\delta x \to 0} \frac{f(x+\delta x, y) - f(x,y)}{\delta x}\]

Q: Notation. \(f_x = \frac{\partial f}{\partial x}, f_{xy} = \frac{\partial^2 f}{\partial y \partial x}\)
A: Second case is interpreted as \((f_x)_y\), do x first then y.

Q: Theorem. If f has continous second partial derivatives, then \(f_{xy} = f_{yx}\)
A: This is an applied course. No proof needed.

Q: Theorem. Chain rule for partial derivatives \[\mathrm{d}f = \frac{\partial f}{\partial x} \mathrm{d}x + \frac{\partial f}{\partial y}\mathrm{d}y\]
A: \[\delta f = f(x+\delta x, y+\delta y) - f(x,y) = f(x+\delta x, y+\delta y) - f(x+\delta x,y) + f(x+\delta x,y) -f(x,y) = f(x+\delta x, y)\delta y + o(\delta y) + f(x,y)\delta x + o(\delta x) = (f_y(x,y) + o(1))\delta y + o(\delta y) + f_x(x,y)\delta x + o(\delta x)\]
By taking the limit as \(\delta x, \delta y \to 0\)we obtain the chain rule

Q: Theorem. Multi-variable implicit differentiation. Given \(F(x,y,z) = c\), then \[\frac{\partial z}{\partial x} = -\frac{(\partial F)/(\partial x)}{(\partial F)(\partial z)}\]
A: \[\mathrm{d}F = \frac{\partial F}{\partial x}\mathrm{d}x + \frac{\partial F}{\partial y}\mathrm{d}y + \frac{\partial F}{\partial z}\mathrm{d}z\]
\[\frac{\partial F}{\partial x} = \frac{\partial F}{\partial x} \frac{\partial x}{\partial x} + \frac{\partial F}{\partial y}\frac{\partial y}{\partial x} + \frac{\partial F}{\partial z}\frac{\partial z}{\partial x} = 0\]
\[\frac{\partial F}{\partial x} + \frac{\partial F}{\partial z} \frac{\partial z}{\partial x} = 0 \implies \frac{\partial z}{\partial x} = -\frac{(\partial F)/(\partial x)}{(\partial F)(\partial z)}\]

Q: Theorem. Differentiation under the integral sign \[\frac{\mathrm{d}}{\mathrm{d}x} \int_0^{b(x)} f(y,c(x)) \mathrm{d}y = f(b,c)b'(x) + c'(x) \int_0^b \frac{\partial f}{\partial c}\mathrm{d}y\]
A: Prove by using the definition of derivative and switching limits and integrals to get \[\frac{\partial I}{\partial c} = \int_0^b \frac{\partial f}{\partial c}\mathrm{d}x\]
In general if \(I(b(x), c(x)) = \int_0^{b(x)} f(y,c(x)) \mathrm{d}y\), then by the chain rule we have \[\frac{\mathrm{d}I}{\mathrm{d}x} = \frac{\partial I}{\partial b}\frac{\mathrm{d}b}{\mathrm{d}x} + \frac{\partial I}{\partial c} \frac{\mathrm{d} c}{\mathrm{d}x} = f(b,c)b'(x) + c'(x)\int_0^b\frac{\partial f}{\partial c} \mathrm{d}y\]

Q: Definition. Eigenfunction
A: An eigenfunction of the derivative operator is a function that is unchanged, up to a multiplicative scaling by the eigenvalue, under the action of the operator. That is, \[\frac{df}{dx} = \lambda f(x)\]

Q: Proposition. \(e^{(x_1 + x_2)} = e^{x_1} e^{x_2}\) using integrals
A: \[x_1 + x_2 = \int_1^{exp(x_1)} \frac{dy}{y} + \int_1^{exp(x_2)}\frac{dy}{y} = \int_1^{exp(x_1)} \frac{dy}{y} + \int_{exp(x_1)}^{exp(x_1)exp(x_2)} \frac{du}{u} = \int_1^{exp(x_1)exp(x_2)} \frac{dy}{y}\]
Also \[x_1 + x_2 = \int_1^{exp(x_1 + x_2)} \frac{dy}{y}\]
So the result follows

Q: Definition. Seperable equation
A: A first order differential equation is seperable if it can be manipulated into the following form \[q(y) dy = p(x)dx\] in which case the solution can be found by integration

Q: Definition. Exact equation
A: \(Q(x,y)\frac{dy}{dx} + P(x,y) = 0\) is an exact solution iff the differential form \(Q(x,y)dy + P(x,y)dx\) is exact, i.e. there exists a function \(f(x,y)\) for which \(df = Q(x,y) dy + P(x,y)dx\). We can show that this means \[ \frac{\partial P}{\partial y} = \frac{\partial Q}{\partial x}\]

Q: Proposition. How to solve general form of a first-order linear differentual equation?
A: \[a(x)\frac{dy}{dx} + b(x)y = c(x) \implies \frac{dy}{dx} + p(x)y = f(x) \]
\[\textbf{Multiply through by integrating factor } \mu(x): \mu (x) = \exp \left[\int^x p(u)du\right]\]
\[\textbf{Now integrate both sides as the LHS is the result of a product rule.}\]

Q: Definition. Simply connected domain
A: A domain \(\mathcal{D}\) is simply connected if it is connected and any closed curve in \(\mathcal{D}\)can be shrunk to a point in \(\mathcal{D}\) without leaving \(\mathcal{D}\)

Q: Theorem. If \(\frac{\partial P}{\partial Y} = \frac{\partial Q}{\partial x}\) through a simply-connected domain \(\mathcal{D}\), then \(P \mathrm{d}x + Q \mathrm{d}y\) is an exact differentual of a single-valued function in \(\mathcal{D}\)
A: If the equation is exact then the solution is simply f = constant, and we can find f by integrating \(\frac{\partial f}{\partial x} = P\) and \(\frac{\partial f}{\partial y} = Q\)

Q: Definition. Equilibrium/ fixed point
A: A constant solution \(y = c\). This corresponds to \(\frac{dy}{dt} = 0\)

Q: Definition. Stability of fixed point
A: An equilibruim is stable if whenever y is deviated slightly from the constant solution y = c, \(y \to c\) as \(t \to \infty\). An equilibrium is unstable if the deviation grows.

Q: Example. Pertubation analysis to determine stability
A: Suppose \(y = a\) is a fixed point of \(\frac{dy}{dt} = f(y,t)\), so \(f(a,t) = 0\). Write \(y = a + \varepsilon (t)\) where \(\varepsilon\) is a small pertubation from a. Putting this into the differential equation we have \[\frac{d \varepsilon}{dt} = \frac{dy}{dt} = f(a+ \varepsilon, t) = f(a,t) + \varepsilon \frac{\partial f}{\partial y} (a,t) + O(\varepsilon^2) = \varepsilon \frac{\partial f}{\partial y} (a,t) + O(\varepsilon^2)\]
Since \(\varepsilon\) is very small we can use this approximation called the linearization of the differential equation \[\frac{\mathrm{d}\varepsilon}{\mathrm{d}t} \cong \varepsilon \frac{\partial f}{\partial y}\]

Q: Definition. Autonomous system
A: A system in the form \(\dot{y} = f(y)\), where the derivative is only dependent on y

Q: Definition. The logistic equation
A: Simple model of population dynamics. \(\frac{dy}{dt} = (\alpha - \beta)y - \gamma y^2 \implies \frac{dy}{dt} = ry(1-\frac{y}{Y})\)
When the population is small we have \(\dot{y} \approx ry\) , so when r>0 population grows exponentially until stable equilibrium Y is reached

Q: Definition. Discrete Equations
A: Might have a model in the form \(x_{n+1} = \lambda x_n(1-x_n)\), discrete logistic equation.
Can be derived from the continuous equation with a discrete approximation. \[\frac{y_{n+1} - y_n}{\Delta t} = ry_n(1 - \frac{y_n}{Y}) \]
\[y_{n+1} = y_n + r \Delta t y _n(1-\frac{y_n}{Y}) = (1+r\Delta t)y_n\ \left [1- (\frac{r \Delta t}{1 + r \Delta t}\frac{y_n}{Y}) \right]\]
Write \(\lambda = 1 + r \Delta t\) and \(x_n = (\frac{r \Delta t}{1 + r \Delta t})\frac{y_n}{Y}\)
\[x_{n+1} = \lambda x_n(1-x_n)\]

Q: Theorem. How to solve the general form of a second order differential equation
A: i) Find the complementary functions which satisfy the homogeneous equation
ii) Find a particular solution that satisfies the full equation

Q: Example. How to solve \(y'' - 4y' + 4y = 0\)
A: We get \(e^{2x}\) as a solution twice, these aren't linearly independent so we only have one basis function of the solution space, but we need a second one as it is a second order equation.
We can perform detuning. We can consider \(y'' - 4y' + (4 - \varepsilon^2)y = 0\) as \(\varepsilon \to 0\). We obtain \(\lambda^2 - 4\lambda + 4 - \varepsilon^2\)as the characteristic equation. We get \(\lambda = 2 \pm \varepsilon\). So \(y = e^{2x}[Ae^{\varepsilon x} + Be^{-\varepsilon x}]\)
Taking the limit as \(\varepsilon \to 0\) and using Taylor expansion we get \(y = e^{2x}[(A+B) + \varepsilon x (A-B) + O(A \varepsilon^2, B \varepsilon^2)]\). Solve for A and B and we get that the solution after taking the limit is \[y \to e^{2x}(\alpha + \beta x)\]

Q: Definition. Linear independence of functions
A: \[\sum_{i=1}^N c_i f_i(x) = 0 \implies (\forall i)c_i = 0\]

Q: Definition. Wronskian
A: Given a differential equation with solutions \(y_1, y_2\), the Wronskian is the determinant \[W(x) = \left | \begin{matrix} y_1 & y_2 \\ y_1' & y_2' \end{matrix}\right |\]

Q: Definition. Independent solutions
A: Two solutions are independent solutions of the differentual equation if and only if the Wronskian is non-zero for some x.
So \(\mathbf{Y_1}\) and \(\mathbf{Y_2}\) are linearly independent as vectors in the phase space for some x.

Q: Definition. Phase space
A: For an nth order differential equation we need the solution vector \[\mathbf{Y(x)} = (y(x), y'(x), \cdots, y^{n-1}(x))\] which lies in the n-dimonsional phase space. So for each x we get a point Y(x). We can view this as the intial conditions for our differential equation.

Q: Theorem. Abel's Theorem. Given an equation \(y'' + p(x)y' + q(x)y = 0\), either \(W = 0\) for all x, or \(W \neq 0\) for all x. Iff two solutions are independent for some particular x, then they are independent for all x
A: Note that \(W = y_1 y_2' - y_2y_1'\). Then \(W' + P(x)W = 0 \implies W(x) = W_0 e^{-\int P dx}\). Since the exponential function is never zero, either \(W_0 = 0\) in which case \(W = 0\) or \(W_0 \neq 0\) and so \(W \neq 0\).

Q: Definition. Equidimensional equation
A: An equation is equidimensional if it has the form \[ax^2y'' + bxy' + cy = f(x)\]

Q: Proposition. Variation of parameters. \[\frac{d^2 y}{dx^2} + p(x)\frac{dy}{dx} + q(x)y = f(x)\] with independent complementary function \(y_1 \text{ and } y_2\)
A: Define the solution vector \(Y(x) = \left ( \begin{matrix} y(x) \\ y'(x)\end{matrix}\right)\)
Since the complementary functions are linearly independent, they form a basis for the solution space. So write the particular integral as \[Y_p(x) = u(x)Y_1(x) + v(x)Y_2(x)\]
Then write out equations for the \(y_p(x)\) and \(y_p'(x)\) components. Differentiating the second equation and adding the correct multiples of each equation to this gives \[u'y_1' + v'y_2' = f\]
Then we can differentiate the equation for \(y_p(x)\) and get that \(u' y_1 + v' y_2 = 0\)
Now we combine these equation and need to solve \[\left ( \begin{matrix} y_1 & y_2 \\ y_1' & y_2' \end{matrix} \right) \left ( \begin{matrix} u' \\ v'\end{matrix} \right) = \left ( \begin{matrix} 0 \\ f \end{matrix} \right)\]
Which we can do by taking inverse matrices etc. to get \[u' = - \frac{y_2}{W} f \text{ and } v' = \frac{y_1}{W}f\]
Combine this with the equation for \(y_p(x)\) to get the final solution.

Q: Definition. Dirac delta function. \[\delta(x) = \lim_{\epsilon \to 0}D(x; \epsilon)\]
A: 1. \(\delta(x) = 0, \forall x \neq 0\)
2. \(\int_{-\infty}^{\infty} \delta(x) dx = 1\)
3.for all functions g(x) that are continuous at x = 0 we have
\(\int_{-\infty}^{\infty} g(x) \delta(x) dx = \lim_{\epsilon \to 0}\int_{-\infty}^{\infty} g(x) D(x ;\epsilon) dx = g(0)\)

Q: Definition. Heaviside step function
A: \[H(x) = \int_{-\infty}^x \delta(t) dt\]It is 0 for \(x<0\), 1 for \(x>0\) undefined at x=0

Q: Proposition. Higher order discerete equations
A: For some difference equation of the form \(ay_{n+2} + by_{n+1} + cy_n\) the complementary functions take the form \(y_n \propto k^n\). Similar rules to second order differential equations apply for everything

Q: Definition. Ordinary and singular points of \[p(x)y'' + q(x)y' + r(x)y = 0\]
A: Ordinary point: If both \(q(x)/p(x)\) and \(r(x)/p(x)\) have Taylor series around the point. Otherwise it is a singular point.
If \(x = x_0\) is a singular point but we can rewrite the equation as \[P(x)(x - x_0)^2 y'' + Q(x)(x - x_0)y' + R(x)y = 0\]
Where \(Q(x)/P(x)\) and \(R(x)/P(x)\) have Taylor series around \(x = x_0\), then it is a regular singular point.
Otherwise it is an irregular singular point.

Q: Theorem. Fuchs' Theorem. If \(x = x_0\) is an ordinary point of \[p(x)y'' + q(x)y' + r(x)y = 0\] then there are two linearly independent solutions of the form \[y = \sum_{n=0}^{\infty} a_n(x-x_0)^n\]What happens if it is a regular singular point?
A: Then there is at least one solution of the form \[y =\sum_{n = 0}^{\infty} a_n(x - x_0)^{n + \sigma}\]This is an example of a Frobenius series.

Q: Proposition. Possibilities when expanding around a regular single point \(x = x_0\) using Frobenius series
1) \(\sigma_2 - \sigma_1\) is not an integer
2)\(\sigma_2 - \sigma_1\) is a non-zero integer
3) \(\sigma_1 = \sigma_2 = \sigma\)
A: 1) Two linearly independent solutions \[y_1 = (x-x_0)^{\sigma_1} \sum_{n=0}^{\infty} a_n(x-x_0)^n\]\[y_2 = (x-x_0)^{\sigma_2}\sum_{n=0}^{\infty} b_n (x-x_0)^n\]2)One solution involving the larger root \(\sigma_2\)
\[y_1 = (x-x_0)^{\sigma_2}\sum_{n=0}^{\infty} a_n (x-x_0)^n\]Second solution of the form \[y_2 = (x-x_0)^{\sigma_1}\sum_{n=0}^{\infty} b_n(x-x_0)^n + cy_1 \ln (x-x_0)\]3) Same as 2) except \(c \neq 0\), it is necessary to have the log term whereas in the second case we do not always need it

Q: Definition. Directional derivative of f in the direction of \(\bf \hat{s}\)
A: \[\frac{df}{ds} = \mathbf{\hat{s}} \cdot \nabla f\]

Q: Definition. Gradient vector \(\nabla f\) of the function f
A: Satisfies \(\frac{df}{ds} = \mathbf{\hat{s}} \cdot \nabla f\) for all unit vectors \(\bf \hat{s}\)

Q: Definition. Hessian matrix
A: The matrix of second derivatives \[\bf H \equiv \left ( \begin{matrix}f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{matrix}\right ) = \nabla \nabla f\]

Q: Definition. Positive-definite and negative-definite matrices
A: A symmetric matrix H is positive definite if \[xHx^T > 0\]For all non-zero row vectors x.

Q: Proposition. Criteria for the Hessian to be positive\negative-definite using eigenvalues
A: \(\bf H\) positive definite \(\iff \forall \lambda_i > 0\)
\(\bf H\) negative definite \(\iff \forall \lambda_i <0\)
If all have mixed signs but are non-zero then the point in a saddle point. If any of the eigenvalues are zero then we need further analysis.

Q: Definition. Signature of Hessian matrix
A: The pattern of the signs of the ordered subdeterminants of its leading principal minors. For a function of n variables, \(f(x_1, x_2 ,\dots , x_n)\), these subdeterminants are \[f_{x_1,x_1}, \left | \begin{matrix} f_{x_1x_1} & f_{x_1x_2} \\ f_{x_2x_1} & f_{x_2x_2}\end{matrix} \right | , \left | \begin{matrix} f_{x_1x_1} & f_{x_1x_2} & f_{x_1x_3} \\ f_{x_2x_1} & f_{x_2x_2} & f_{x_2x_3} \\ f_{x_3x_1} & f_{x_3x_2} & f_{x_3x_3}\end{matrix} \right | , \dots , |\bf H_n| = |H|\]Sylvester's criterion says that:
\(\bf H\) positive definite \(\iff\) signature is \(+, +, \dots , +\)
\(\bf H\) negative definite \(\iff\) signature is \(-,+, \dots , (-1)^n\)

Q: Proposition. What do the countours look like locally at minima, maxima and saddle points?
A: Write \(\mathbf{H} = \left ( \begin{matrix}\lambda_1 & 0 \\ 0 & \lambda_2 \end{matrix}\right)\), where the diagonal is the eigenvalues which we assume are non-zero.
If we write \(\mathbf{x = x_0} + (\zeta, \eta)\)
Then in the vicinity of \(\bf x_0 \): \(f(\mathbf{x}) = f(\mathbf{x_0}) + \frac{1}{2}(\lambda_1 \zeta^2 + \lambda_2 \eta^2)\)
The countours of f therefore locally satisfy: \(\lambda_1 \zeta^2 + \lambda_2 \eta^2 = \text{constant}\)
At a maximum or minimum the eigenvalues all have the same sign so the countours are locally elliptical.
At a saddle point they have opposite signs so the contours are locally hyperbolic.

Q: Proposition. How to solve a coupled system of linear differential equations of the form \[\mathbf{\dot{Y} = MY + F}(t)\]
A: 1. Write \(\bf Y = Y_c + Y_p\) where the complementary solution satisfies \(\bf \dot{Y}_c = MY_c\)
2. Look for a complementary solution of the form \(\mathbf{Y_c =v}e^{\lambda t} \), where \(\bf v\) is a constant vector. For this to satisfy the equation above we must have \(\mathbf{Mv} = \lambda \mathbf{v}\). So \(\bf v\) is an eigenvector of M and \(\lambda\) is the associated eigenvalue.
3. Find a particular solution which satisfies the full system of forced equations.

Q: Definition. Effective Potential.
A: \[V_{\text{eff}}(r) = V(r) + \frac{ml^2}{2r^2}\]
Extra \(\frac{ml^2}{2r^2}\) term is the angular momentum barrier.
It stops the particle getting too close to the origin, since there it must pay a heavy price in effective energy
We get \[E = \frac{1}{2}m\dot{r}^2 + V_{\text{eff}}(r)\]
The effective potential energy is the real potential energy, together with a contribution form the angular kinetic energy