| SOCR ≫ | BPAD Website ≫ | BPAD GitHub ≫ |
Biomedical Physics with Applications to Disease (BPAD)
Chapter 4: Complex Particle Systems
BPAD Team + SOCR
January 2024
Some simple physical systems permit reliable deterministic modeling, or static model-based inference. For instance, for a single mass attached to a spring, we can use Newton’s second law (\(F = m\times a\)) to model the motion of the system. Similarly, for a single atom it is possible to use the quantum-mechanical equivalent of \(F = m\times a\), the Schrödinger equation whose wavefunction solution captures the dynamics of the periodic atomic motion. Fitting a linear model of an outcome using observed data represents yet another example of a static representation of intrinsically stochastic processes that generate the observations.
The situation of modeling and representing the dynamics of more complex systems involving many particles is much more difficult, and sometimes impossible all together. Consider the example of modeling the number of particles in a cubic millimeter of blood (\(1mm^3\)), which may contain a number of different constituents with different volume, concentration, and
For instance, the hemoglobin concentration in blood depends on many factors (e.g., hormonal levels) and is expected to be around \(15 g/dl\pm 3 g/dl\) representing about \(1.4\times 10^{15}\) hemoglobin molecules in \(1 mm^3\) of blood. Whereas the corresponding concentration and number of water molecules \(H_2O\) in \(1 mm^3\) of blood are \(1 g/cm^3 \pm 0.01g/cm^3\) and \(3.3 \times 10^{19}\), respectively. Of course, the number of oxygen atoms attached to each of the four iron sites in the hemoglobin molecule may vary widely depending on the location of the blood sample, oxygenated and deoxygeneted blood travels outward from and into the lungs, respectively.
Modeling a 3D translational motion of each particle may require three equations representing the the position in each of the basic directions. Suppose at time \(t\) the force acting on a particle is \(F(t)=(F_x(t), F_y(t), F_z(t)):\mathbb{R}^+ \to \mathbb{R}^3\). Then, using Newton’s second law, \(F=a\times m\), and first-order Taylor series expansion on a time interval \([t, t + \Delta t]\), the velocity of the particle, \(v(t)=(v_x(t), v_y(t), v_z(t))\), changes according to the three equations
\[\left | \begin{aligned} v_x (t + \Delta t) & = v_x (t) + \frac{F_x \Delta t}{m} \\ v_y (t + \Delta t) & = v_y (t) + \frac{F_y \Delta t}{m} \\ v_z (t + \Delta t) & = v_z (t) + \frac{F_z \Delta t}{m} \\ \end{aligned} \right. .\]
The corresponding three equations representing the \(x\), \(y\), and \(z\) changes of the position of the particle, \({\bf{x}}(t)=({\bf{x}}_x(t), {\bf{x}}_y(t), {\bf{x}}_z(t))\), are expressed as a second order Taylor series (in terms of the velocity \(v(t)\) and the acceleration, \(a(t)=\frac{F(t)}{m}\))
\[\left | \begin{aligned} {\bf{x}}_x(t + \Delta t) &= {\bf{x}}_x(t) + v_x (t)\Delta t + \frac{F_x (t)(\Delta t)^2}{2m} \\ {\bf{y}}_x(t + \Delta t) &= {\bf{y}}_x(t) + v_y (t)\Delta t + \frac{F_y (t)(\Delta t)^2}{2m} \\ {\bf{z}}_x(t + \Delta t) &= {\bf{z}}_x(t) + v_z (t)\Delta t + \frac{F_z (t)(\Delta t)^2}{2m} \\ \end{aligned} \right. .\]
For small \(\Delta t\), the second term can be dropped and we can use again a first order approximation for the position \({\bf{x}}(t)\). Solving these equations for each particle requires a number of addition and multiplication operations, e.g., for \(10^{19}\) particles (\(H_2O\) molecules in \(1cm^3\) of blood), the system solution may require over \(10^{20}\) arithmetic operations per time interval. Modern GPU chips can perform less than a petaflop operations, i.e., (\(10^{15}\)) floating point operations per second (FLOPS). This suggests that solving the position calculations solely for the water molecules in a tiny blood sample may require \(10^5\) seconds (over a day).
Computational complexity is just one issue that makes complex system modeling challenging. The physics of the processes represent another limitation that sometimes leads to deterministic chaos, even in relatively simple systems. For instance, identically configured experimental systems that are initialized using slightly different starting conditions may become unstable and generate progressively divergent subsequent behaviors even over a brief period of time. This is also known as the butterfly effect.
The tracing, modeling, forecasting, or prediction of the behavior of such chaotic systems is impossible, and sometimes may not be necessary. In certain situations, the overall properties of a system are more interesting than the specific instantiations or results. For instance, we may be interested in the distribution of the results, e.g., reporting average number of molecules, overall variation of pressure, skewness of the concentration, kurtosis of the speed, and so on.
1 Double Pendulum
A double pendulum is a system of a pair of interconnected pendula attached at their ends of their arms. The dynamics of the double pendulum system chaotic and sensitivity to its initial conditions.
Let’s model this system parameterized by the angles \(\theta_1\) and \(\theta_2\) between each arm and the vertical axis. Both arms are identical and of equal length, \(l\), and mass, \(m\). Assume that the uniform gravity acceleration is \(g\), the arm moment of inertia is \(I = \frac{1}{12}ml^2\) about the arm’s central point of mass, and the origin of the Cartesian coordinate system is the base point of suspension of the first pendulum arm. Then, the positions of the centers of mass of each arm, the points \(P_1=(x_1, y_1)\) and \(P_2=(x_2, y_2)\), can be expressed in terms of the parameters \(\theta_i\):
\[{\begin{aligned}x_{1}&={\frac {l}{2}}\sin \theta _{1} \\ y_{1}&=-{\frac {l}{2}}\cos \theta _{1} \\ x_{2}&=l\left(\sin \theta _{1}+{\tfrac {1}{2}}\sin \theta _{2}\right)\\y_{2}&=-l\left(\cos \theta _{1}+{\tfrac {1}{2}}\cos \theta _{2}\right) \end{aligned}}.\]
The system Lagrangian can be expressed as
\[\begin{aligned} L &= \underbrace{ \overbrace{\frac {1}{2}m\left(v_{1}^{2}+v_{2}^{2}\right)}^{linear\ kinetic\ energy\\ of\ the\ center\ of\ mass} +\overbrace{\frac {1}{2}I\left({{\dot {\theta }}_{1}}^{2}+{{\dot {\theta }}_{2}}^{2}\right)}^{rotational\ kinetic\ energy\ around\\ the\ center\ of\ mass\ for\ each\ arm}}_{Total\ kinetic\ energy}- \underbrace{mg\left(y_{1}+y_{2}\right)}_{potential\ energy}\\ &={\tfrac {1}{2}}m\left({{\dot {x}}_{1}}^{2}+{{\dot {y}}_{1}}^{2}+{{\dot {x}}_{2}}^{2}+{{\dot {y}}_{2}}^{2}\right)+{\tfrac {1}{2}}I\left({{\dot {\theta }}_{1}}^{2}+{{\dot {\theta }}_{2}}^{2}\right)-mg\left(y_{1}+y_{2}\right)\end{aligned}.\]
The time derivatives are denoted by \(f'(t)\equiv \dot {f}(t)\) for each time-dependent variable.
In terms of the bivariate parameterization (\(\theta_1,\theta_2\)) of the centers of mass positions of the two arms, the Lagrangian is:
\[L= \frac {1}{6}ml^{2}\left({{\dot {\theta }}_{2}}^{2}+4{{\dot {\theta }}_{1}}^{2}+3{{\dot {\theta }}_{1}}{{\dot {\theta }}_{2}}\cos(\theta _{1}-\theta _{2})\right)+ \frac {1}{2}mgl\left(3\cos \theta _{1}+\cos \theta _{2}\right).\]
To derive the equations of motion for the double pendulum, we will express the two generalized momenta in terms of the angle parameters. Note that the total energy (kinetic plus potential) is conserved, but the momenta are not conserved:
\[\begin{aligned} p_{\theta_{1}} &= {\frac {\partial L}{\partial {{\dot {\theta }}_{1}}}}={\tfrac {1}{6}}ml^{2}\left(8{{\dot {\theta }}_{1}}+3{{\dot {\theta }}_{2}}\cos(\theta _{1}-\theta _{2})\right)\\ p_{\theta _{2}}&={\frac {\partial L}{\partial {{\dot {\theta }}_{2}}}}={\tfrac {1}{6}}ml^{2}\left(2{{\dot {\theta }}_{2}}+3{{\dot {\theta }}_{1}}\cos(\theta _{1}-\theta _{2})\right)\end{aligned}.\]
Solving the above equations for \((\dot{\theta}_{1},\dot{\theta}_{2})\), we get:
\[\begin{aligned} {{\dot {\theta }}_{1}}&={\frac {6}{ml^{2}}}{\frac {2p_{\theta _{1}}-3\cos(\theta _{1}-\theta _{2})p_{\theta _{2}}}{16-9\cos ^{2}(\theta _{1}-\theta _{2})}} \\ {{\dot {\theta }}_{2}}&={\frac {6}{ml^{2}}}{\frac {8p_{\theta _{2}}-3\cos(\theta _{1}-\theta _{2})p_{\theta _{1}}}{16-9\cos ^{2}(\theta _{1}-\theta _{2})}} \end{aligned} .\]
Hence, the four double pendulum equations of motion showing the time evolution of the chaotic system given its current state can be expressed in terms of the time-varying parameters \(\theta_1\) and \(\theta_2\):
\[\begin{aligned} {{\dot {\theta }}_{1}}&={\frac {6}{ml^{2}}}{\frac {2p_{\theta _{1}}-3\cos(\theta _{1}-\theta _{2})p_{\theta _{2}}}{16-9\cos ^{2}(\theta _{1}-\theta _{2})}}\\ {{\dot {\theta }}_{2}}&={\frac {6}{ml^{2}}}{\frac {8p_{\theta _{2}}-3\cos(\theta _{1}-\theta _{2})p_{\theta _{1}}}{16-9\cos ^{2}(\theta _{1}-\theta _{2})}} \\ {{\dot {p}}_{\theta _{1}}}&={\frac {\partial L}{\partial \theta _{1}}}=-{\tfrac {1}{2}}ml^{2}\left({{\dot {\theta }}_{1}}{{\dot {\theta }}_{2}}\sin(\theta _{1}-\theta _{2})+3{\frac {g}{l}}\sin \theta _{1}\right)\\ {{\dot {p}}_{\theta _{2}}}&={\frac {\partial L}{\partial \theta _{2}}}=-{\tfrac {1}{2}}ml^{2}\left(-{{\dot {\theta }}_{1}}{{\dot {\theta }}_{2}}\sin(\theta _{1}-\theta _{2})+{\frac {g}{l}}\sin \theta _{2}\right) \end{aligned}.\]
The simulation below shows the path traversed by the end-point of the
second pendulum arm. The simulation below solves the differential
equations above by numerical integration using the method
odeintr::integrate_sys(). We can explore the effects of
different initial conditions and parameter settings on the system
dynamics, as well as, run the simulation multiple times with the same
initial conditions.
# Setup configuration
library(tidyverse)
library(plotly)
# 1. Constants and initial conditions
G <- 9.807 # acceleration due to gravity, in m/s^2
L1 <- 1.0 # length of pendulum 1 (m)
L2 <- 1.0 # length of pendulum 2 (m)
M1 <- 4.0 # mass of pendulum 1 (kg)
M2 <- 4.0 # mass of pendulum 2 (kg)
parms <- c(L1, L2, M1, M2, G)
# initial conditions
theta1 <- 30 # initial angle theta of pendulum 1 (degree)
w1 <- 5.0 # initial angular velocity of pendulum 1 (degrees per second)
theta2 <- 60 # initial angle theta of pendulum 2 (degree)
w2 <- -5.0 # initial angular velocity of pendulum 2 (degrees per second)
state <- c(theta1, w1, theta2, w2)*pi/180 #convert degree to radians
x1 <- L1*sin(state[1])
y1 <- -L1*cos(state[1])
x2 <- L2*sin(state[3]) + x1
y2 <- -L2*cos(state[3]) + y1
df <- tibble(x1, y1, x2, y2)
plot_ly(df) %>%
# Add arm of first pendulum
add_segments(xend=x1, yend=y1, x=0, y=0, type="scatter", mode="lines",
line = list(width = M1), name="Arm 1") %>%
# Attach the arm of the second pendulum
add_segments(xend=x2, yend=y2, x=x1, y=y1, type="scatter", mode="lines",
line = list(width = M1), name="Arm 2") %>%
# Add theta1
add_text(x = 0.1, y = -0.5, text = "\U1D703<sub>1</sub>", name="\U1D703<sub>1</sub>") %>%
# Add theta2
add_segments(xend=x1, yend=y1-1, x=x1, y=y1, type="scatter", mode="lines",
line = list(width = 1, color="black"), name="Theta2 Angle") %>%
add_text(x = 0.6, y = -1.3, text = "\U1D703<sub>2</sub>", name="\U1D703<sub>2</sub>") %>%
# Add anchor base point
add_trace(x=0, y=0, type="scatter", mode="markers",
marker = list(size = 20), name="Anchor Base") %>%
# add joint point
add_trace(x=x1, y=y1, type="scatter", mode="markers",
marker = list(size = 20), name="Joint Point") %>%
# add last (free) end-point
add_trace(x=x2, y=y2, type="scatter", mode="markers",
marker = list(size = 20), name="Free End Point") %>%
# add first center of mass point (arm 1)
add_trace(x=x1/2, y=y1/2, type="scatter", mode="markers",
marker = list(size = 10), name="Arm 1 Center of Mass") %>%
add_text(x = x1/2+0.4, y =y1/2, text = "(x<sub>1</sub>,y<sub>1</sub>)", name="(x<sub>1</sub>,y<sub>1</sub>)") %>%
# add second center of mass point (arm 2)
add_trace(x=(x1+x2)/2, y=(y1+y2)/2, type="scatter", mode="markers",
marker = list(size = 10), name="Arm 2 Center of Mass") %>%
add_text(x = (x1+x2)/2+0.4, y =(y1+y2)/2, text = "(x<sub>2</sub>,y<sub>2</sub>)", name="(x<sub>2</sub>,y<sub>2</sub>)") %>%
# Add annotation for gravity acceleration
add_annotations(ax = (x1+x2)/2, ay =(y1+y2)/2-0.1, axref = "x", ayref = "y",
arrowhead=2, text= "<i>m g</i>", showarrow=T,
x=(x1+x2)/2, y =(y1+y2)/2-1) %>%
# add rectangular field of view and layout the scene
layout(title = 'Double Pendulum - Initial State',
shapes = list(list(type = "rect",
line = list(color = "blue"),
opacity = 0.1,
x0 = "-2", x1 = "2", xref = "x",
y0 = -3, y1 = 1, yref = "y")),
legend = list(orientation = 'h'),
yaxis = list(scaleanchor = "x"))Numerical integration for solving the differential equations and animating the solutions over a period of 60 seconds (s).
# install.packages("gganimate")
library(tidyverse)
library(gganimate)
# 1. Constants and initial conditions
G <- 9.807 # acceleration due to gravity, in m/s^2
L1 <- 1.0 # length of pendulum 1 (m)
L2 <- 1.0 # length of pendulum 2 (m)
M1 <- 4.0 # mass of pendulum 1 (kg)
M2 <- 4.0 # mass of pendulum 2 (kg)
parms <- c(L1, L2, M1, M2, G)
# initial conditions
theta1 <- 20.0 # initial angle theta of pendulum 1 (degree)
w1 <- 5.0 # initial angular velocity of pendulum 1 (degrees per second)
theta2 <- 180.0 # initial angle theta of pendulum 2 (degree)
w2 <- -5.0 # initial angular velocity of pendulum 2 (degrees per second)
state <- c(theta1, w1, theta2, w2)*pi/180 #convert degree to radians
# Partial derivatives calculator
pdes <- function(state, t){
L1 <- parms[1]
L2 <- parms[2]
M1 <- parms[3]
M2 <- parms[4]
G <- parms[5]
dydx <- rep(0,length(state))
dydx[1] <- state[2]
del_theta <- state[3] - state[1]
den1 <- (M1 + M2)*L1 - M2*L1*cos(del_theta)*cos(del_theta)
dydx[2] <- (M2*L1*state[2]*state[3]*sin(del_theta)*cos(del_theta) +
M2*G*sin(state[3])*cos(del_theta) +
M2*L2*state[4]*state[4]*sin(del_theta) -
(M1 + M2)*G*sin(state[1]))/den1
dydx[3] <- state[4]
den2 <- (L2/L1)*den1
dydx[4] <- (-M2*L2*state[4]*state[4]*sin(del_theta)*cos(del_theta) +
(M1 + M2)*G*sin(state[1])*cos(del_theta) -
(M1 + M2)*L1*state[2]*state[2]*sin(del_theta) -
(M1 + M2)*G*sin(state[3]))/den2
return(dydx)
}
# 2. ODE Integration to find solutions of equations of motion
# start duration and step_size parameters control the time dynamics of the double pendulum
# We can run the pendulum for 60 seconds and record the end-point position at increments of 0.1 seconds.
# install.packages("odeintr")
library(odeintr)
ode_solver <- integrate_sys(pdes, init = state, duration = 60, start = 0, step_size = 0.1)
# 3. Use the theta parameters obtained in step 2 via integration
# compute the spatial coordinates for both pendulums (x_i,y_i)
x1 <- L1*sin(ode_solver[, 2])
y1 <- -L1*cos(ode_solver[, 2])
x2 <- L2*sin(ode_solver[, 4]) + x1
y2 <- -L2*cos(ode_solver[, 4]) + y1
df <- tibble(t = ode_solver[,1], x1, y1, x2, y2, group=1)
# 4. Plot the DF containing the spatial positions of the pendulum arms for time
# and animating the pendulum dynamics
y_offset <- 1.5 # shift upwards the base point (or set to zero)
# pl <- ggplot(df)+
# geom_segment(aes(xend=x1, yend=y1+y_offset), x=0, y=y_offset, size=M1, col="green")+
# geom_segment(aes(xend=x2, yend=y2+y_offset, x=x1, y=y1+y_offset, size=M2, col="red"), show.legend = FALSE)+
# geom_point(size=5, x=0, y=y_offset, col="purple")+
# geom_point(aes(x1, y1+y_offset), col="yellow", size=5)+
# geom_point(aes(x2,y2+y_offset), col="blue", size=5)+
# scale_y_continuous(limits=c(-2,2))+
# scale_x_continuous(limits=c(-2,2))+
# theme(legend.position="none") +
# ggraph::theme_graph()+
# labs(title="{frame_time} s")+
# transition_time(t)
# double_pendulum <- animate(pl, nframes=nrow(df), fps=25)
# double_pendulum
# 5. Trace the path of the end-point of the second pendulum
# install.packages("transformr")
library(transformr)
tmp <- select(df, t, x2, y2)
trail <- tibble(x=c(sapply(1:5,function(x) lag(tmp$x2,x))),
y=c(sapply(1:5,function(x) lag(tmp$y2,x))),
t=rep(tmp$t,5)) %>%
dplyr::filter(!is.na(x))
pl <- ggplot(df)+
geom_path(data=trail, aes(x,y+y_offset, group=t), colour="blue", size=0.5)+
geom_segment(aes(xend=x1, yend=y1+y_offset), x=0, y=y_offset, size=M1, col="green")+
geom_segment(aes(xend=x2, yend=y2+y_offset, x=x1, y=y1+y_offset, size=M2, col="red"), show.legend = FALSE)+
geom_point(size=5, x=0, y=y_offset, col="purple")+
geom_point(aes(x1, y1+y_offset), col="yellow", size=5)+
geom_point(aes(x2,y2+y_offset), col="blue", size=5)+
scale_y_continuous(limits=c(-2,2))+
scale_x_continuous(limits=c(-2,2))+
theme(legend.position="none") +
ggraph::theme_graph()+
labs(title="Time {frame_time} s")+
transition_time(t) +
shadow_mark(colour="grey", size=0.2, exclude_layer = 2:6)
double_pendulum <- animate(pl, nframes=nrow(df), fps=25) #, rewind=TRUE, start_pause=3)
double_pendulum
anim_save("C:/Users/Dinov/Desktop/DoublePendulumAnime.gif", double_pendulum)
# 
2 Gas Molecules in a Box
Suppose a box ov volume \(V\) contains only \(N\) gas particles. If the box volume is split into a disjoint union of two complementary parts, \(V= V_1 \dot\bigcup V_2\), let \(p\) be the probability that a single one of the \(N\) particles is inside the first volume \(V_1\). Then, the residual probability that the particle is in the second volume \(V_2\) will be \(q=1-p\).
Assuming each part of the box is indistinguishable from any other one part of equal volume, the probability \(p=\frac{V_1}{V}=\frac{V_1}{V_1\cup V_2}\). Similarly, \(q = \frac{V_2}{V}\).
Suppose \(N\) gas particles are released in the box of volume \(V\). The distribution of the \(N\) particles within the two sub-volumes can be taught as Binomial experiment with the number of particles in \(V_1\) denoted by \(n_1\) and the remaining particles present in \(V_2\) labeled by \(n_2\), \(N=n_1 + n_2\). At any point in time, the chance that exactly \(n\) of the total \(N\) particles are found in volume \(V_1\) is given by the binomial probability distribution: \[P(n; N,p) = \frac{N!}{n! (N − n)!}p^n (1 − p)^{N−n} .\] Try the binomial probability distribution calculator.
Note that \(0\leq n\leq N\) and there a value of \(n\) maximizing the likelihood function, i.e., the probability mass function \(P\) is maximized at the median \(\lfloor Np \rfloor\) or \(\lceil Np \rceil\). Away from the median, the binomial probability rapidly decreases to zero.
The binomial probability distribution provides a model-based characterization of the complex process of quantifying the spread of the aprticles in the box volume, without having to explicitly label which particles are in volume \(V_1\) and which are in the complement area, \(V_2\). v. Of course, the number of gas particles in \(V_1\) is not static and is expected to change dynamically, however, its expected to fluctuated about the distribution point of gravitational balance (its median), where \(P\) is maximized. For instance, an experiment with a total of \(N = 6\) particles generating an observation of only two particles in the left half of the box (\(V_1\equiv V_2\)) has a change of occurrence equal to \[P(n=2; N=6, p=0.5) = \frac{6!}{2! (4)!}(0.5)^2 (0.5)^4=0.234375 .\]
On average, we would expect to observe \(3\) particles in either half of the box, and this maximum chance corresponds to: \[P(n=3; N=6, p=0.5) = \frac{6!}{3! (3)!}(0.5)^3 (0.5)^3=0.3125 .\]
## [1] 0.234375## [1] 0.3125# Binomial simulation with a sample of size 100,000 (random repetitions)
N <- 6
p <- 0.5
binomial_draws <- rbinom(100000, N, p)
expected_value <- N * p
variance <- N * p * (1 - p)
stdev <- sqrt(variance)
a <- list(
text = paste0("Binomial(N=",N, ",p=", p,") Sumulation of Size 100,000"),
xref = "paper",
yref = "paper",
yanchor = "bottom",
xanchor = "center",
align = "center",
x = 0.5,
y = 1,
showarrow = FALSE
)
densities <- table(binomial_draws)/100000
p_binomial <- plot_ly() %>%
add_histogram(binomial_draws,
xbins=list(start = 0, end = N+1, size = 1),
marker=list(color="#1f78b4",line=list(color = "white",width=1))) %>% add_trace(x = c(1:7)-0.5, y = densities, type="scatter", mode = "markers", yaxis = "y2", name = "Density") %>%
layout(title = "", # legend = list(orientation = 'h'),
annotations = a,
xaxis = list(title = "Value"),
yaxis = list(title = ""),
yaxis2 = list(overlaying = "y", side = "right"))
p_binomialSuppose at the initial state of the experiment, all particles are constrained only in the left half of the box. removing the constraint (partitioning barrier) the availability of additional space typically leads to particle diffusion. The particles tend to proliferate and occupy the entire box. The state of all particles remaining in the left half of the box once the partitioning is removed is quite unlikely.
The more orderly state of the system prior to the removal of the partition keeping the particles confined to the left half of the box, becomes more chaotic (uncertain, random) as the particles are freed to occupy the entire box. The system spontaneously approaches a much more random arrangement with approximately half of the particles occupying each half of the box. Of course, the actual number of particles in \(V_1\) and \(V_2\) will vary around the expected value \(\frac{N}{2}\), but the law of large numbers dictates that the average \(\langle n \rangle\) converges to the true population mean \(\frac{N}{2}\). The dynamics of this behavior is shown via the simulation below depicting that the average number of particles in each half approaches the expected value.
means <- array()
for (i in 1:length(binomial_draws)) {
means[i] <- mean(binomial_draws[1:i])
}
plot_ly(y=~means, type="scatter", mode="lines")The system equilibrium is characterized as the static state where the observed average numbers of particles remains constant over time, even though individual instances may fluctuate around the average value corresponding to the equilibrium state.
3 Laws of Thermodynamics
The general study of energy, its transfers and transformations is called thermodynamics. The thermodynamics laws provide the axiomatic basis for the field of thermodynamics. These principles include conservation of energy and the limitations of converting energy to useful work. Life itself represents a diverse array of biological machines that organize matter and energy and alter the system entropy. The universal nature of thermodynamic laws makes them useful to model a wide range of physical processes and environmental systems.
The thermodynamics principles (laws) quantify physical observations, e.g., temperature, energy, momentum, spin, and entropy, and describe dynamic systems and their equilibria. For instance, these formulations express the dependencies between thermodynamic work and heat, and quantify likelihoods of certain relationship configuration, e.g., impossibility of constructing a realistic perpetual motion machine work infinitely without some external energy input.
The three fundamental thermodynamics laws are paired with another zeroth law of thermodynamics defining thermal equilibrium using temperature - Given two systems, \(S_1\) and \(S_2\), each in thermal equilibrium with another third system, \(S_3\), then,\(S_1\) and \(S_2\) must be in a thermal equilibrium with each other.
The first law of thermodynamics postulates that as energy passes into or out of a system (manifested as work, heat or matter transfer), the internal system energy changes accordingly to conserve the total energy. The second law of thermodynamics establishes that the total sum of the entropies of all interacting systems can’t decreases, i.e., the entropy of closed systems increases, just like a broken egg can’t spontaneously reformulate itself in it’s original form. The third law of thermodynamics dictates that the system entropy approaches a constant value (typically near zero) as the temperature approaches absolute zero.
3.1 Energy
Energy is a quantitative property of a system, or an environment, that commonly transferred between constituents of the system, e.g., to perform work, grow or heat an organism, or alter the environment itself, e.g., change physico-chemical properties of matter. All living organisms exchange heat with the environment and each other.
The total energy of a system is the sum of two forms of energy - potential and kinetic energies. The kinetic energy tracks the movement of an object, or the aggregate motion of all of its components. The potential energy reflects the potential of an object to generate motion as a function of state or position.
There are a number of specific energy types that reflect specific combinations of potential and kinetic energy. For instance, the mechanical energy represents the sum of translational and rotational kinetic and potential energies of a system. Nuclear energy is the combined potential energies of an atomic nucleus from either the nuclear weak or strong nuclear forces. Sound wave energy consists of the kinetic and potential energies of a medium that is due to a propagating mechanical sound wave. Rest energy is the potential energy due solely to an object’s rest mass, and thermal energy is the kinetic energy of the microscopic motion of particles.
The relationship between living organisms, their interactions with the ambient environment, and their interactions (dependencies) may be examined by modeling the transfer and exchange of energy, e.g., heat flow, mass transport, or chemical reactions. More specific biophysical examples of energy transfer include the rate of photosynthesis, amino acid synthesis, and the production of metabolic heat. Energy is the key concept in the first law of thermodynamics, which delineates the system balance of energy input, transfer, production, and output.
Often times, specific environmental conditions may promote species survival by reduction, increase, or alteration of the flow of one special type of energy (heat). Climate adaptations and heat transfer, balance, and use affect the distribution and activities of all life forms, including plants and animals.
Recent experiments at the International Space Station (ISS) demonstrated life adaption by exposing bacteria to extreme dark/bright and inhospitable environment 250 miles above the Earth (where radiation levels exceed 200 times those on the Earth’s surface). Without any protection from the powerful cosmic ultraviolet, gamma, and x-rays, many bacteria survived these conditions during the three-year experiment, which explored the ability of living organisms handle large amounts of radiation exposure during potential interplanetary travel. The results show that the deinococcus radiodurans, a type of extremophilic bacterium, may survived at least three years of exposure to extreme conditions in earth’s outer space. The observations suggest that while deinococcus bacteria in the outer layers died, they dead cells protected bacterial in the inner layers from irreparable DNA damage. Aside the physical protective barrier formed by dead cells in the masses, the bacteria were remarkably resistant to DNA damage from stellar radiation. The deinococcus radioduran contains up to 10 redundant copies of genes coding for proteins that find and repair damaged DNA segments. For comparison, mammalian cells carry about two copies of DNA and most bacterial cells carry a single copy. Transcribing additional DNA copies of important genes may require higher energy input, but also yields higher reproducibility of the corresponding repairing proteins.
3.2 Temperature
Temperature is a continuous measure of the molecular vibrations within a system. The zeroth law of thermodynamics states that given that two systems \(S_1\) and \(S_2\) are individually in thermal equilibrium with a third system \(S_3\) (e.g., a thermometer), then \(S_1\) and \(S_2\) are in a thermal equilibrium with each other.
Commonly used temperature scales are Fahrenheit (\(^oF\)), Celsius (\(^oC\)), and Kelvin (\(K\)), all are based on the unique triple point of water, \(273.16 K\). The graph below shows a phase diagram of water (\(H_2O\)) against the pressure \(P\) the temperature \(T\). Solid, liquid, and vapor state regions occur at different combinations of temperature and pressure. The unique water triple point (intersection of the sublimation, freezing (fusion), and vaporization curves) illustrates the state where all three phases occur simultaneously. This property is the basis of the definition of temperature scales relative to a unique physical value.
A pressure-temperature phase diagram for any substance shows the conditional dependencies between variable quantities (e.g., pressure, temperature, volume) controlling the distinct thermodynamic phases (solid, liquid or gaseous states) and their coexistence at triple point of the phase equilibrium. For instance, the triple point of intersection of the sublimation, fusion and vaporization curves for water \(H_2O\) is shown in the figure and the short animation illustrates the change in the distribution of relative amounts of solid, liquid and gas amounts of water.
The Gibbs phase rule determining the number of independently manipulable variables (e.g., temperature, pressure) is given by \(F=C-P+2\), where where \(F\) is the number of degrees of freedom, \(C\) is the number of components, and \(P\) is the number of phases.
This video is generated using a workflow extending the \(Mathematica^{(R)}\) protocol in Neil C. Hendren “Pressure-Temperature Phase Diagram for Water”, Wolfram Demonstrations Project, Published: December 15 2017, accessed February 2, 2022.
Exampled of some common temperatures in different scales are shown in the table below.
| Temperature Points | \(^oF\) | \(^oC\) | \(K\) |
|---|---|---|---|
| Absolute zero | -459.67 | -273.15 | 0.00 |
| Water freezing point | 32.00 | 0.00 | 273.15 |
| Triple point of water | 32.018 | 0.01 | 273.16 |
| Boiling point of water | 212.00 | 100.00 | 373.15 |
Linear temperature scale transformations:
\[^oF = 1.8 ({^o C}) + 32.0, \ \ \ \ \ K = {^oC}+ 273.15 . \]
3.3 Thermal Equilibrium
A system is in thermodynamic equilibrium if its temperature is the same as the temperature of its ambient environment. A system is in steady state if its temperature is fixed (static not dynamic) over time. Note that the time scale is important, as short and long-term steady-states have different interpretations. Large objects, e.g., solar systems or even human bodies, are often in non-steady (transient) states because reaching an equilibrium may be along oscillatory process with temperature fluctuating around one or more equilibria and because of external influences, e.g., interstellar radiation, diurnal air temperature cycles, atmospheric and solar wind conditions and other time-varying processes.
All real and physically observable systems are open, not “closed”, which permits the exchange of mass and energy with their ambient surroundings. For instance, life forms exchange heat, energy, and nutrients with their environments during the normal metabolic processes. Closed systems exhibit no sch mass, energy or matter exchanged with their surroundings. For reproducibility reasons, many biophysical experiments aim to design environmental conditions approximating closed systems where causal-effects (inputs-conditions-outputs) may be studied. Whereas most biophysical systems are open and affected by external factors. Reversible processes are an idealized thermodynamic processes conserving the energy across time. In practice, such processes are extremely difficult to construct as all realistic biophysical processes involving energy exchange exhibit heat loss (e.g., due to friction), as dictated by the second law of thermodynamics.
3.4 First Law of Thermodynamics (System Energy)
System energy conservation is the principle of the first law of thermodynamics. It postulates that the change in internal system energy (within the system boundary), \(\Delta U\), is equal to the difference between the heat added to the system, \(Q\), and the output work done by the system to the surroundings, \(W\),
\[\underbrace{\Delta U}_{energy\ change} = \underbrace{Q}_{added\ heat} - \underbrace{W}_{work\ output}.\]
Another formulation of the first law is in terms of infinitesimal changes and reversible processes:
\[dU = \delta Q - \delta W.\]
The slightly different differential notations, \(d\) and \(\delta\), correspond to the exact and inexact (loose or imperfect) differentials. The inexact differential reflects the fact that the heat \(Q\) the work \(W\) are path-dependent (hence we use \(\delta Q\) and \(\delta W\)), whereas the system internal energy \(U\) is path-independent (and we use \(dU\)). A more formal definition of inexact derivative is based on vector calculus. Given a vector \(F\), \(\delta f = F \cdot d r\) is an inexact differential if there is no function \(f\) such that \(F =\nabla f\).
Recall that then expressing the values of a given vector field \(F\) in terms of the partial derivatives of another function \(f\), the fundamental theorem of calculus for line integrals the (the multivariate analogue of the univariate antiderivative principle) requires path independence. There is no unique representation of a multivariate antiderivative for inexact differentials, since their variation is inconsistent along different paths in the configuration space. Of course, in univariate (1D) calculus, there is only one path in between two real points, which is defined by a function path independence. The increase of the domain complexity in multivariate processes requires this formulataion of inexact differentials.
For instance, given an idealized gas system with internal energy \(U_A\) at state \(A\), the system state variables controlling the system are pressure, \(P\), volume,\(V\), and temperature, \(T\). At a new system state \(B\), the system energy will be a function \(U_B = f (P_B,V_B, T_B)\), and the system state change is described by \(\Delta U = Q-W\).
The work and heat variables can’t be uniquely expressed in terms of \(P\), \(V\) and \(T\) and by itself, the magnitude of the difference for the change of state, \(dU\), can’t be explicated in terms of heat and work. The total change in \(W\) and \(Q\) depends on the particular (state-space) path the system goes through as it transforms from \(U_A \to U_B\). This trajectory affects the integration of \(\delta W\) and \(\delta Q\) between \(U_A\) and \(U_B\). The inexact loose differential, \(\delta\), is used to represent this path dependence of the \(Q\) and \(W\) changes.
The first law sign conventions can change, in this formulation, \(dU>0\) when when the new energy flows into the system, and as heat \(Q\) flows into the system, it adds to its internal energy. Similarly, completing system work \(W\) (output) subtracts from its internal energy, \(U\).
There are alternative energy units used track energy transfer in different studies (e.g., metabolic activity, atomic reactions, engineering applications). The calorie is a common energy unit used for tracking biological measurements. One calorie is the energy necessary to raise one gram of water from \(14.5^oC\) to \(15.5^oC\) at atmospheric pressure. In general scientific research, the International System of Units (SI) measured in Joule (\(J\)) is used:
\[\underbrace{J}_{Joule}={\frac {kg\cdot m^{2}}{s^{2}}} = N\cdot m=Pa\cdot m^{3}=W\cdot s=C\cdot V=\Omega \cdot A^{2}\cdot s,\]
where the notation is \(kg= kilogram\) (weight), \(m=meter\) (length), \(s=second\) (time), \(N= Newton\), \(Pa=Pascal\) (pressure), \(W=Watt\), \(C=Coulomb\), \(V=volt\), \(\Omega=Ohm\), and \(A=ampere\).
One joule corresponds to the work necessary to move an electric charge of one Coulomb through an electrical potential difference of one volt, which is equivalent to one coulomb-volt (\(C⋅V\)).
The internal energy of a system, \(U\), is often the temperature, the special type of thermal energy. The chemical energy is the energy holding atoms together is itself part of the total system internal energy. The internal thermal energy is the vibrational energy of the molecules and is represents the temperature of the system. Temperature changes may induce structural substance changes, e.g., the increase of internal energy of \(H_2O\) makes the phase of water change between solid, liquid and vapor.
The heat term, \(Q\), in the first law represents the energy in transit during the heat transfer process with heat flowing spontaneously in one direction, from warmer to colder objects. There are four kinds of heat transfer mechanisms:
- Conduction - representing the physical process of thermal energy transport in solids, e.g., heat flow between animals and air.
- Convection - transferring of heat between solids and fluids, or gasses and fluids, that are in contact. Conduction involves particles in close proximity where the circulation of the fluid makes convection different from conduction. Convection is a more complex process involving heat transfer and fluid dynamics.
- Radiation - transmitting energy without the presence of an ambient medium, e.g., in vacuum or in interstellar space. All objects, living and nonliving, radiate some form of energy. The amount and type (i.e., wavelength in the electromagnetic spectrum) of the radiated energy depends on the temperature and physical properties of the energy source. The duality of radiation is characterized as particles (photons) and waves (wave fields), which contrasts the acoustic type waves that propagate only within a medium.
- Evaporation - transporting substances agros phase boundaries, e.g., changing water from a liquid to a gas state. Through skin respiration, water vapor cools animal bodies. The physiology of each species determines the extent of this (evaporation) heat and mass flow process. SImilarly, during transpiration, floral species, e.g., green plants, lose water vapor to the atmosphere through stomata.
The figure below shows the electromagnetic spectrum as a function of frequency (\(f\)) (Hertz or cycles per second) and wavelength (\(\lambda\)) (meters).
In a gas system, for each state variable, \(x\), e.g., pressure, volume or temperature, the heat capacity is \(C_x = \left (\frac{\delta Q}{dT}\right )_x\), where the subindex \(x\) specifies the path of the process and its reversibility for a fixed value of the appropriate state variable.
Work, typically mechanical work, is the product of a distance and a force operating in some system. In general, work is done by the system affecting the surrounding environment and decreasing the system’s energy. For living organisms, organs, or cell systems, work may be mostly associated with externally facing, rather than focused solely on internal chemical reactions internal to the system. For instance, if we study the heart, then we may study of the heart work and it’s impact on the rest of the body.
3.5 Second Law of Thermodynamics (Entropy)
3.5.1 Efficiency
Often, energy is transferred to or from one system to another through heat (temperature) or by work (exerting a force through a distance). This energy may involve energy conversion into other form in other systems. For example, as the body works out and burns calories, heat is transferred into perspiration. Such body work converts the ATP energy into a variety of other forms — an increase in the body’s kinetic or gravitational potential energy, into electrical energy (if we are using a hand/foot crank electric generator, and heat transfer from the body to the surrounding air, etc. Most of the thermal energy transfer is released into the surroundings. For instance, combustion vehicle engines are only about \(25-30\%\) efficient in generating work, which is in stark contrast with the \(90\%+\) efficiency of electric vehicles, see this US DoE Report (Using Natural Gas for Vehicles: Comparing Three Technologies - Gas, Electric, Hydrogen).
The efficiency (\(ε>0\)) of a cycle is defined as the ratio between the absolute value of the work extracted from the cycle (\(|W_{tot}|\)) and the heat that gets into the system (\(|Q_h|\)):
\[ε=\frac{|W_{tot}|}{|Q_h|}.\]
A Carnot cycle is an idealized process composed of a alternating pair of isothermal and another pair of adiabatic (reversable) transformations, expanding or compressing an ideal gas, where there is no energy is lost to mechanical friction. Let’s consider a Carnot cycle connecting two heat reservoirs at temperatures \(T_h\) (hot) and \(T_l\) (low) that have a large thermal capacity, so their temperatures are unaffected by the cycle itself. The system is composed only by the ideal gas that changes its temperature throughout the cycle. The four transformations of a Carnot cycle involve:
- Stage 1: isothermal expansion \(A\rightarrow B\). At this stage, heat is released from the hot reservoir and then is absorbed by the ideal gas particles within the system, causing a rise of the temperature of the system. As the gas particles expand, they push the piston upwards and do work on the surroundings
\[\Delta U_1 = \underbrace{W_1}_{<0} + \underbrace{Q_1}_{>0} = 0 \Longrightarrow Q_1 = -W_1.\] The heat and work at this stage are \[Q_1 = |Q_h|=n R T_h \underbrace{\ln \frac{V_B}{V_A}}_{>0,\ V_B > V_A} > 0,\] \[W_1 = -Q_1 = - nRT_h \ln \frac{V_B}{V_A} < 0,\] where \(|Q_h|\) is the absolute value of the heat that gets into the system from the hot reservoir.
- Stage 2: adiabatic expansion \(B \rightarrow C\). An expansion that continues with no heat exchange between the system and its surroundings, and allows the ideal gas particles to cool, decreasing the temperature of the system.
\[Q_2 = 0 \Longrightarrow \Delta U_2 = W_2.\] The negative energy (expansion work) is \[\Delta U_2 = W_2 = n \underbrace{\int_{T_h}^{T_l} {C_V dT}}_{<0\ as\ T_{l}<T_{h}} < 0.\]
- Stage 3: another isothermal compression \(C \rightarrow D\) where the surrounding does work оп the system releasing heat, with the temperature of the system remaining unchanged. The expressions of the heat are similar to those in stage 1, however they result in reversing the signs of the heat and work, since this is a compression.
\[\Delta U_3 = \underbrace{W_3}_{>0} + \underbrace{Q_3}_{<0} = 0 \Longrightarrow Q_3 = -W_3,\] \[Q_3 = | Q_l| = nRT_l \underbrace{\ln \frac{V_D}{V_C}}_{<0\ as\ V_D<V_C} < 0,\] \[W_3 = -Q_3 = -nR T_l \ln \frac{V_D}{V_C} > 0,\] where \(|Q_l|\) is the amount of the heat going out of the system to the cold reservoir (\(|Q_l|\) is the heat entering the system).
- Stage 4: during the final adiabatic compression \(D \rightarrow A\) stage, there is no heat exchange, yet, the surroundings continue to do work on the system. The induced adiabatic compression raises the temperature of the system and the location of the piston back to its original state before the start of the first stage.
\[Q_4 = 0 \Longrightarrow \Delta U_4 = W_4,\] During adiabatic conditions, the process energy is positive (compression work) \[\Delta U_4 = W_4 = n \underbrace{\int_{T_l}^{T_h} C_V dT}_{>0\ as\ T_{h}>T_{l}} > 0.\]
\[\Delta U_{tot} = \Delta U_1+\Delta U_2+\Delta U_3+\Delta U_4 = 0\] since the path integral over the simple closed curve is trivial \(\oint dU=0\). Yet, the amounts of work and heat are not trivial as \(Q\) and \(W\) are path functions.
\[W_{tot} = W_1+W_2+W_3+W_4=\] \[W_{tot} = - nR \left( T_h-T_l \right) \ln \frac{V_B}{V_A} < 0,\] as \(T_h>T_l\) and \(V_B>V_A\). Negative work implies that the work is done by the system, since the system performs \(PV\)-work by transferring heat from a hot reservoir to a cold one via a Carnot cycle.
The Carnot cycles works in the \(hot \to cold\) direction (\(A → B → C → D → A\)), just like common heat engines do to produce work.
The efficiency of a Carnot cycle is
\[\varepsilon = \frac{Q_3 + Q_1}{Q_1} = 1+ \frac{Q_3}{Q_1}=\\ \frac{nR(T_h−T_l)\ln\left (\frac{VB}{VA}\right )}{nR T_h \ln\left (\frac{VB}{VA}\right )}= \frac{T_h−T_l}{T_h}=1−\frac{T_l}{T_h} < 1,\]
where the process cycle starts from state \(1\) (system at \((P_1,V_1)\), expands to state \(2\) (system at \((P_2,V_2)\)), and compresses back to state 1 (system back to \((P_1,V_1)\)).
This shows that the efficiency of a Carnot cycle is strictly less than \(1.0\), i.e., there are no cycle that can convert \(100\%\) of heat into work it extracts from a hot reservoir. In 1851, Lord Kelvin noted It is impossible for a self-acting machine, unaided by any external agency, to convey heat from one body to another at a higher temperature.
3.5.2 Carnot cycle
This simulation shows the pressure(\(P\)) vs. Volume (\(V\)) diagram for Carnot cycle composed of isothermal and adiabatic processes.
3.5.3 Entropy
Recall from the chapter introduction that it is not even theoretically possible for engines to be 100 percent efficient. This phenomenon is explained by the second law of thermodynamics, which relies on a concept known as entropy. Entropy is a measure of the disorder of a system. Entropy also describes how much energy is not available to do work. The more disordered a system and higher the entropy, the less of a system’s energy is available to do work.
Although all forms of energy can be used to do work, it is not possible to use the entire available energy for work. Consequently, not all energy transferred by heat can be converted into work, and some of it is lost in the form of waste heat—that is, heat that does not go toward doing work. The unavailability of energy is important in thermodynamics; in fact, the field originated from efforts to convert heat to work, as is done by engines.
The equation for the change in entropy, \(ΔS\), is \(ΔS=QT\),where \(Q\) is the heat that transfers energy during a process, and \(T\) is the absolute temperature at which the process takes place. Also, \(Q\) is positive for energy transferred into the system by heat, and negative for energy transferred out of the system by heat. In SI, entropy is expressed in units of Joules per Kelvin (\(J/K\)). If temperature changes during the process, then it is usually a good approximation for small changes in temperature to take \(T\) as the average temperature.
Using the relation \(\frac{Q_3}{T_l} = -\frac{Q_1}{T_h}\), we can express the efficiency of a Carnot cycle as \[\varepsilon = 1+\frac{Q_3}{Q_1} = 1-\frac{T_l}{T_h}.\]
The sum of reversible and irreversible conditions in a Carnot cycle are captured in the equation \(\frac{Q_3}{T_l} + \frac{Q_1}{T_h}=0\). The relative reversible heat exchange is \(\frac{Q_{rev}}{T}\), where each transformation in a Carnot cycle is reversible, and \(\frac{Q_3}{T_l} + \frac{Q_1}{T_h}=0\) can be generalized to a sequence of connected Carnot cycles joining two or more isotherms by integrating across a sequence of Carnot cycles that connects different temperature:
\[\underbrace{\sum_i \frac{Q_{rev}}{T_i}}_{conserved} = 0.\] This illustrates that for a series of connected Carnot cycles, the fraction of heat associated with each reversible stage relative to the temperature at which such heat is exchanged (\(T_i\)), is conserved around the cycle, and it is independent on the actual simple closed path.
We can generalize this conserved state function, called entropy,
\[S = \frac{Q_{rev}}{T},\] to any reversible cycle in a \(PV\) phase-diagram by integrating the infinitesimal quantity \(S\) over the path.
\[dS = \frac{\delta Q_{rev}}{T}.\] In addition to reversible cycles that play role in the heat dynamics and entropy, there are also irreversible cycles. The efficiency of a Carnot cycle \(\varepsilon = 1+\frac{Q_3}{Q_1} = 1-\frac{T_l}{T_h}\) represents the maximum efficiency of an idealized thermodynamic cycle.
Any other suboptimal efficiencies correspond to irreversible cycles lower efficiency smaller, relative to the maximum efficiency of an idealized Carnot cycle. Hence,
\[{1+\frac{Q_3}{Q_1}} =\underbrace{\varepsilon_{irr}}_{irrevirsible\\ efficiency} < \underbrace{\varepsilon_{rev}}_{revirsible\\ efficiency}= {1-\frac{T_l}{T_h}}.\]
This loss in efficiency implies that
\[\frac{Q^{irr}_3}{Q^{irr}_1} < - \frac{T_l}{T_h} \Longrightarrow \frac{Q^{irr}_3}{T_l} + \frac{Q^{irr}_1}{T_h} < 0 \Longrightarrow \sum_i \frac{Q_{irr}}{T_i} < 0,\]
and correspondingly \[\oint \frac{\delta Q_{irr}}{T} < 0.\] And we obtain the general Clausius inequality \[\oint \frac{\delta Q}{T} \leq 0,\]
with \(\oint \frac{\delta Q}{T} \equiv 0\) corresponding to for reversible transformations, and \(\oint \frac{\delta Q}{T} < 0\) for irreversible cycles.
Let’s consider an isolated system undergoing a cycle consisting of an irreversible forward transformation (\(a \rightarrow b\)) and another reversible backward transformation (\(b \rightarrow a\)). The spontaneity of the irreversible expansion process is in contrast with the non-spontaneous reversible backward compression. Then the entropy corresponding to the irreversible part is:
\[0 > \oint \frac{\delta Q_{irr}}{T} \equiv \int_a^b\frac{\delta Q_{irr}}{T} + \int_b^a \frac{\delta Q_{rev}}{T}.\]
Hence, \[\int_a^b \frac{\delta Q_{irr}}{T} + \int_b^a \frac{\delta Q_{rev}}{T} < 0.\] Using \(\delta Q_{irr}=0\) and swapping the limits of the reversible integral,
\[\underbrace{\int_a^b \frac{\delta Q_{rev}}{T}}_{\int_a^b dS = \Delta S} > \underbrace{\int_a^b \frac{\delta Q_{irr}}{T}}_{=0},\] Thus, the equation \(\Delta S > 0\), shows that for an isolated system, any irreversible process has to have an increasing (with time) entropy, which is a reflection of the second law of thermodynamics.
4 Boltzmann Factor
The Boltzmann factor is a physical constant describing the relation between particle concentration and potential energy. It allows us to calculate the amount of energy associated with random thermal motions of particles in a substance. For instance, as the temperature of gas molecules increases, then the average kinetic energy of the molecules also goes up. Think of a foil helium balloon you may buy at room temperature in a store and walk out with in subfreezing weather. The result of this will be a rapid shrinkage of the foil boundary as the energy of the gas inside drops. However, he balloon is probably not leaking and will reconstitute its original fully-inflated shape once it’s back at normal room temperature.
\[Temperature (R)\propto Kinetic\ Energy.\] The sensation of heat is due to the random motion of atoms and molecules that in high entropy states are hitting boundary sensors, e.g., a hand, with high kinetic energy.
The higher the system the temperature, the higher the entropy of these particles and the greater the energy transmission (heat transfer).
A model of an ideal gas equation, \(PV = nRT\equiv n\kappa T\) (universal gas law) connects \(P\), pressure in Pascals, \(V\), volume in \(m^3\), \(n=\frac{Substance\ mass}{mass\ of\ 1\ mole}\), \(R=\kappa {N_A}=8.314\cdot Joule\cdot mol^{-1}\cdot Kelvin^{-1}\), molar gas constant, \(T\), temperature in Kelvin, \(N\), number of molecules, and \(N_A=6.022 \times 10^{23}\ mol^{-1}\), Avogadro constant, and \(\kappa\equiv \kappa_B=\frac{R}{N_A}\) is the Boltzmann constant.
A thermodynamic system at an absolute temperature \(T\) carries an average thermal energy of \(\frac{1}{2}kT\)%, which at room temperature is \(\sim 2.07\times 10^{−21} J=0.013 eV\).
In kinetic theory, the average pressure \(P\) for an ideal gas is:
\[P = {\frac{1}{3}}{\frac{N}{V}}m{\overline {v^{2}}}.\]
Then, the ideal gas law \(PV=N\kappa T\), yields the average translational kinetic energy
\[\frac {1}{2} m \overline {v^2}=\frac {3}{2}\kappa T.\]
The translational motion velocity 3D vector \(v\) has three degrees of freedom, which suggests that per degree of freedom, the average energy is \(\frac {1}{2}\kappa T\).
Generalizing ideal gases to molecular gases requires a modification of the heat capacity as molecules possess additional internal degrees of freedom in addition to the three spatial movement degrees of freedom. For instance, diatomic gases have a total of six degrees of freedom per molecule, reflecting 3 translational, 2 rotational, and 1 vibrational atomic motions. Quantum mechanical limits on the availability of excited states at relevant molecular thermal energy may reduce the degrees of freedom in gas heat capacity at low temperatures.
The Boltzmann factor may also be interpreted in probabilistic terms, reflecting systems in equilibrium at temperature \(T\) that have probability \(P_i\) of occupying a state \(i\) with energy \(E\) weighted by a corresponding Boltzmann factor
\[P_{i}\propto {\frac {e^\left(-{\frac {E}{\kappa T}}\right)}{Z}},\] where \(Z\) is the corresponding partition function of the thermodynamic state variables. In a statistical sense, the entropy \(S\) of an isolated system at thermodynamic equilibrium can be expressed in terms of the number of distinct microscopic states \(W\) available to the system for a fixed total energy \(E\):
\[S=\kappa \ln W.\] The notation \(W\) come from Wahrscheinlichkeit, the German term for probability, defined on certain physical systems. Naturally, physical systems tend to to be observed more frequently in higher probability states, and correspondingly, observed less frequently in less probable states. The interpretation of the entropy increase is just the expectation of inevitable transition from less probable to more probable states. At microscopic levels, the entropy fluctuates, as most systems are dynamically moving between states according to some transition probability map.
There is a direct relation between entropy and information. The statistical definition of entropy is just the negative information, or information is just negative entropy . Information is a concept used as a limit on the amount of signal communication (information) could be transmitted via a noisy communication channel.
Given a random variable \(X\), the information-theoretic definition of entropy of \(X\) is the average level of uncertainty relative the likelihoods of the process possible outcomes. For instance, the entropy of a discrete random variable \(X\) with possible outcomes \(\{x_1, x_2, \cdots, x_{n}\}\) corresponding to the following probabilities \(\{p_1=P(x_{1}),p_2=P(x_{2}),\cdots, p_n=P(x_{n})\}\) is:
\[H(X)=-\sum_{i=1}^{n}{P(x_{i})\log P(x_{i})}=-\sum_{i=1}^{n}{p_i\log (p_i)},\]
In maximum likelihood estimation (MLE), we try to maximize the expectation of the log-likelihood function, i.e., the Kullback-Leibler information function. For identifiable models, the maximum of the log-likelihood represents the true unknown parameter value.
Also, ergodic theory, developed by Boltzmann to study the behavior of physical systems, such as molecules of a gas, with constant energy can be formulated in terms of \(X(t)\), the state of the system at time \(t\), \(I_{(B)}\) is the indicator function of the state subset \(B\), \(E(x)\), the system energy in state \(x\), \(\mathcal{S}\) representing the set of states where \(E(x) = E(X(0))\), and \(W\), the volume of \(\mathcal{S}\). Ergodicity suggest that for any (well-behaved) set of states \(A\),
\[\lim_{T\to\infty}\left \{\frac{1}{T}\int_{t=0}^{T}{\mathbf{I}_{(X(t) \in A)} dt} \right \} = \int_{\mathcal{S}}{\mathbf{I}_{(x \in A)} \frac{1}{W} dx}.\]
In other words, the average along a path \(X(t)\) (LHS) is equal to the average over the region of constant energy (RHS). The composite word ergodic comes from ergon (work, energy, erg is a unit of energy) and hodos (path). Most ergodic theorems dictate that classes of processes (functions of processes, distributions, functors) are ergodic.