Produce a Lunar Eclipse Video

Background

This is a short post recording how I processed the lunar eclipse video fragments to get a moon eclipse dynamic illustration.

The Moon Eclipse event was on 2022/11/8 from UTC 08:02:17 to 13:56:08.

The video fragments are recorded with my A7M3 camera with a FE85mmF1.8 lens on a small tripod.

All are commodity hobbyist devices. No star tracker or other accessories.

Therefore I need to do image processing to get a desirable result.

Dissemble raw video into pngs

ffmpeg -hide_banner -i input.mp4 raw%04d.png

Direct frame splitting. Do notice that raw frames could occupy much more storage than raw video.

One can use python-ffmpeg lib to capture and process each frame on the go with VideoCapture class, which would save lots of storage space.

The tradeoff is hard to view and test, as the process is highly sequential. This also increases the complexity of parallelization.

Or One can just run multiple processes on multiple video fragments

Anyway, I use python to do the image processing. Opencv and numpy are used.

import cv2 as cv
import numpy as np
from concurrent.futures import ProcessPoolExecutor # for parallel processing
resolution = [640,360]

**Note: Python has PIL for **

Use HoughCircles() for rough detection

The moon moved up in a trajectory in my video as there is no star tracker in my toolbox.

Therefore I need to locate the moon and crop the frame accordingly.

def rough_detect(imagefile):

    image = cv.imread(cv.samples.findFile(imagefile), cv.IMREAD_COLOR)

    gray = cv.cvtColor(image, cv.COLOR_BGR2GRAY)
    
    rows = gray.shape[0]
    circles = cv.HoughCircles(gray, cv.HOUGH_GRADIENT, 1, rows / 8,
                               param1=100, param2=10,
                               minRadius=1, maxRadius=0)

	if circles is not None:
		circles = np.uint16(np.around(circles))
		# Only assess the first result
		circle = circles[0, :][0]
		center = [circle[0], circle[1]]
		radius = circle[2]
		
		# Crop it here or after
		# image = image[center[1]-resolution[1]//2:center[1]+resolution[1]//2, center[0]-resolution[0]//2:center[0]+resolution[0]//2]
	else:
		raise Exception("No Moon!")

	# Illustrate the image if you prefer
	# cv.imshow(image) 

	return image, circle

There would be pixel-wise errors in Hough circle detection, depending on the image quality and complexity.

Croping the image directly would generate frequent displacement and vibration.

So I only use hough circle detection as a rough bound estimate for a per-pixel scan.

Search area to find precise coordinates (Parallelly)

To get the precise boundary of the moon I just search in the reference bound given by hough circle detection.

You can see that the bottom-left of the moon is solid, therefore I am going to find the precise left and bottom boundaries of the moon.

To help reduce workload, the radius estimate from the previous process is also helpful.

Though the estimate is not accurate, a 10x10 scan is good enough to find the precise value.

def pixel_scan(image,circle):
	(thresh, gray) = cv.threshold(image, 128, 255, cv.THRESH_BINARY | cv.THRESH_OTSU)
	Y = -1
	X = -1
	delta = 5
	for x in range(circle[0]-circle[2] - delta,circle[0]-circle[2]+delta):
		for y in range(circle[1] - delta, circle[1] + delta):
			if gray[y,x] > 200:
				X=x
				break
		if X != -1:
			break
	for y in reversed(range(circle[1]+circle[2] - delta, circle[1] + circle[2] + delta)):
		for x in range(circle[0] - delta, circle[0] + delta):
			if gray[y,x] > 200:
				Y = y
				break
		if Y != -1:
			break
	if X == -1 or Y == -1:
		raise Exception("No Boundaries")
	else:
		return X, Y

Thanks to the previous processing, I only need to scan a minimal area for the result.

Extract and Repack the video

Now I can crop the image and put the moon at the center.

Notice that the radius is also an unstable estimate, which should only be used as a bound estimate.

A wrong radius would make all frames deviate by several pixels, but an unstable radius would make them vibrate.

I set a fixed radius according to previous processing.

radius = 100

def generate_img(image,X,Y,filename):
	image = image[Y - radius - resolution[1] // 2: Y - radius + resolution[1] // 2, X + radius - resolution[0] // 2:X + radius + resolution[0] // 2]
	try:
		cv.imwrite(filename, image)
	except:
		raise Exception("Writing File failed")

Then I use ffmpeg again to concatenate the frames

ffmpeg -hide_banner -hwaccel cuda -framerate 60 -pattern_type glob -i 'frame*.png' -c:v hevc_nvenc -pixel_fmt yuv420p out.mp4

One can tweak the parameters of ffmpeg using this post as a reference.

Now I got a perfectly stable moon eclipse illustration, that coming from my camera and my mind.