Distance, Bearing, and Position on a Sphere

Coordinates in One Minute

Every point on Earth can be described by two angles. Latitude (written φ) measures how far north or south of the equator a point is, from 90° S to 90° N. Longitude (written λ) measures how far east or west it is of the prime meridian through Greenwich, England, from 180° W to 180° E. In calculations, north and east are positive and south and west are negative, so a longitude of 96.7970° W becomes −96.7970.

People read coordinates in degrees, but the trigonometric functions in almost every programming language expect radians. The conversion is one line, and it has to happen before any of the formulas below will work.

One more fact is worth memorizing. A degree of latitude is about 111 km (69 miles) everywhere on Earth. A degree of longitude is the same length only at the equator. Lines of longitude converge toward the poles, so the length of a degree shrinks with the cosine of the latitude. At 60° north it is half as long, about 56 km (35 miles), and at the poles it is zero.

Great-Circle Distance

The shortest path between two points on a sphere runs along a great circle: a circle whose center is the center of the sphere. The equator is a great circle, and so is every line of longitude. There are three common ways to calculate the length of that path, each with its own trade-offs.

The Haversine Formula

The haversine formula is named after an old trigonometric function, the "half versed sine," which is simply sin²(θ / 2). It takes the two latitudes and the differences in latitude and longitude, and returns the distance along the surface.

The first line produces a number between 0 and 1 that describes how far apart the points are. The second line turns it into an angle at the center of the Earth and multiplies by the radius to get a distance.

The Spherical Law of Cosines

The same distance can be written in a single line using the spherical law of cosines.

Mathematically, this gives exactly the same answer as haversine. The difference is numerical. When two points are very close together, the value inside the arccos is extremely close to 1, and tiny rounding errors in the computer's arithmetic are magnified. Haversine behaves better for very small distances, while the law of cosines is shorter to write and to read.

The Equirectangular Approximation

Over a small area, the Earth is nearly flat. The equirectangular approximation takes advantage of that: it treats the two points as corners of a right triangle and applies the Pythagorean theorem, after shrinking the east-west side by the cosine of the latitude.

It needs one cosine and one square root, which makes it fast, and it is fine for short distances. It falls apart over long distances, where the curvature it ignores adds up, and near the poles, where lines of longitude crowd together.

The Earth Is Not a Sphere

All three formulas above assume a perfect sphere. The real Earth bulges at the equator and is flattened at the poles. The standard model of that shape, used by GPS, is the WGS-84 ellipsoid, with an equatorial radius of 6,378.137 km and a flattening of 1/298.257223563. That makes the polar radius about 21 km shorter than the equatorial one.

Measuring distance on an ellipsoid is harder, and there is no single-line formula for it. Vincenty's formulae, published in 1975, repeat a calculation until the answer stops changing, and reach sub-millimeter accuracy on the ellipsoid. Their known weakness is that the iteration can fail to converge when the two points are nearly antipodal, meaning on almost exactly opposite sides of the planet. Karney's algorithm, published in 2013, solves the same problem and always converges.

How much does the choice of model matter? Spherical formulas are typically within about half a percent of the ellipsoidal answer. Whether half a percent is negligible or unacceptable depends entirely on what the number is for.

Bearing: Which Way to Go

Initial Bearing

A bearing is a compass direction measured clockwise from true north: 0° is north, 90° is east, 180° is south, and 270° is west. The initial bearing, also called the forward azimuth, is the direction to face at the start of a great-circle path.

The second line matters. The atan2 function returns angles from −180° to 180°, so a direction slightly west of north comes out as a small negative number. Adding 360 and taking the remainder normalizes it to the 0–360 range a compass uses.

The Bearing Changes as You Travel

On a great circle, the bearing is not constant. Follow the shortest path from New York to London and you set off heading northeast, at about 51°, and arrive heading east-southeast, at about 108°. The initial and final bearings differ on every great-circle route except those that run along the equator or due north or south.

A path that holds one constant bearing the whole way is called a rhumb line. It is easier to steer, and it is slightly longer than the great circle between the same two points.

Comparing Two Bearings

Bearings wrap around at 360°, so plain subtraction can give the wrong answer. A heading of 350° and a heading of 10° are 20° apart, not 340°.

Destination Point: Where You End Up

The reverse problem is just as useful: given a starting point, a bearing θ, and a distance d, where do you end up? The first step is to convert the distance into an angle at the center of the Earth, the angular distance δ = d / R. The new latitude and longitude follow from that.

The result is in radians and needs converting back to degrees. If the path crosses the 180° meridian, the longitude also needs wrapping back into the −180° to 180° range.

A Worked Example: Dallas to Oklahoma City

Take downtown Dallas, Texas (32.7767° N, 96.7970° W) as the start and downtown Oklahoma City, Oklahoma (35.4676° N, 97.5164° W) as the end. The haversine formula, step by step:

That is 306.45 km, or 190.42 miles. Here is how the other methods compare on the same pair of points:

• Law of cosines: the same 306.45 km, as expected.
• Equirectangular approximation: 306.46 km. That is about 3 meters longer, a 0.001% difference.
• WGS-84 ellipsoid: 305.77 km (190.00 miles). The sphere overestimates by 679 meters, or 0.22%.

For the bearing, atan2 returns −12.28°, which normalizes to an initial bearing of 347.72°, or north-northwest. The final bearing on arrival is 347.32°. Over a route this short and this close to due north, the heading drifts by less than half a degree.

The destination formula closes the loop. Start in Dallas, head out at 347.72° for 306.45 km, and it returns 35.4676° N, 97.5165° W. That is within a few meters of the Oklahoma City coordinates, and the leftover comes from rounding the bearing and distance to two decimal places.

A Longer Route for Contrast

From New York (40.7128° N, 74.0060° W) to London (51.5074° N, 0.1278° W), the picture changes. Haversine gives 5,570 km and the ellipsoid gives 5,585 km, so this time the sphere comes up about 15 km short. The error from the spherical model can run in either direction. The initial bearing is 51.2°. The equirectangular shortcut, which was 3 meters off between Dallas and Oklahoma City, is off by about 4.5% here.

Comparing the Methods

Each method is a tool with its own balance of accuracy, speed, and complexity. Which trade-off is right depends on the distances involved and on how the result will be used.

Try It: Distance and Bearing Calculator

The calculator below comes pre-filled with the Dallas to Oklahoma City example. The first tab shows the spherical (haversine) distance, the WGS-84 ellipsoid distance, the difference between the two, and the initial bearing. The second tab finds a destination point from a start, a bearing, and a distance. Try the New York and London coordinates to watch the gap between the two Earth models change.

Common Pitfalls

• Degrees versus radians: The most common mistake of all. Passing 32.7767 straight into a sine function treats it as 32.7767 radians and produces a wrong answer without any error message.
• Latitude and longitude order: People usually say "latitude, longitude," but some data formats, including GeoJSON, store longitude first. Swapped coordinates can put a point on another continent.
• The antimeridian: Longitude wraps at ±180°. Points at 179° E and 179° W are 2° apart, not 358°. The trigonometric formulas handle this on their own, but anything that subtracts longitudes directly, including the equirectangular approximation, needs the difference wrapped into the −180° to 180° range first.
• The poles: At a pole, longitude has no meaning and every direction points the same way: from the North Pole, every heading is south. Bearings and flat approximations become unreliable nearby.
• Straight-line distance is not driving distance: A great-circle distance ignores roads entirely, and the drive is almost always longer. Distance along roads comes from a road network, which is a different kind of calculation.
• False precision: GPS readings themselves carry several meters of error. The fifth decimal place of a degree of latitude is about 1.1 meters, so extra decimal places beyond that do not mean extra accuracy.