NTP timestamp on Jpeg images

[Marianna Smidth Buschle]

I have no experience with the NTP timestamp part.

But I have successfully appended custom metadata to the EXIF of JPEG.

In this way I'm not depend on gstreamer for extracting the metadata again.


You need a jpegenc and a jifmux:

g_string_append_printf(pipe_desc,
"jpegenc name=%s! jifmux name=%s! ",
JPEG_ENCODER_NAME, JIF_MUX_NAME);

Then add a pad probe to the jpegenc:

//add pad probe to jpeg encoder
enc= get_elem(bin, JPEG_ENCODER_NAME);
if(enc!= NULL) {
GstPad*pad= gst_element_get_static_pad(enc, "src");
gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BUFFER,
(GstPadProbeCallback) jpeg_encoder_cb, NULL, NULL);
gst_object_unref(pad);
}
gst_object_unref(enc);

The callback to attach the metadata using the tagsetter:

GstPadProbeReturn
jpeg_encoder_cb(GstPad*pad, GstPadProbeInfo*info, gpointeruser_data)
{
GstTagSetter*tagsetter;
GstElement*mux;
structPan355_RPC_Datapos0, pos1, tar0, tar1;
QtecGeometryPoint3dcurrent_xyz, target_xyz;
gdoubletof;
QtecGeometryPoint3dcam_loc;
structPan355Intrinsicsintrinsics;
structQtecMatrix*cam_rot= NULL;
gintwidth= -1, height= -1;
gdoubleratio_x= 1, ratio_y= 1, cx, cy, fx, fy, f_mm, px_size_x, px_size_y;
//QTEC_DEBUG("jpeg_encoder_cb");
pos0.axis= tar0.axis= 0;
pos1.axis= tar1.axis= 1;
mux= get_elem(bin, JIF_MUX_NAME);
if(!mux)
returnGST_PAD_PROBE_OK;
tagsetter= GST_TAG_SETTER(mux);
//get position
if(!get_axis_pos(&pos0) || !get_axis_pos(&pos1) ||
!get_xyz_pos(pos0.angle, pos1.angle, 0, &current_xyz) ||
!get_axis_params(&tar0) || !get_axis_params(&tar1) ||
!get_xyz_pos(tar0.angle, tar1.angle, 0, &target_xyz)) {
gst_tag_setter_add_tags(tagsetter, GST_TAG_MERGE_REPLACE,
GST_TAG_ARTIST, "msb at qtec.com",
NULL);
gst_object_unref(mux);
returnGST_PAD_PROBE_OK;
}
get_cam_tilt_offset(&tof);
get_cam_loc(&cam_loc);
get_cam_intrinsics(&intrinsics);
cam_rot= get_cam_rot_matrix();
GstCaps*caps= gst_pad_get_current_caps(pad);
if(caps!= NULL) {
GstStructure*structure= gst_caps_get_structure(caps, 0);
if(structure!= NULL) {
gst_structure_get_int(structure, "width", &width);
gst_structure_get_int(structure, "height", &height);
}
gst_caps_unref(caps);
}
//re-scale principal point and pixel size based on MJPEG size
if(width> 0)
ratio_x= FULL_HD_WIDTH/ width;
if(height> 0)
ratio_y= FULL_HD_HEIGHT/ height;
cx= intrinsics.cx/ ratio_x;
cy= intrinsics.cy/ ratio_y;
fx= intrinsics.fx/ ratio_x;
fy= intrinsics.fy/ ratio_y;
f_mm= intrinsics.fx* DECIMATED_PIXEL_SIZE(SONY_PIXEL_SIZE) / 1000;
px_size_x= DECIMATED_PIXEL_SIZE(SONY_PIXEL_SIZE) * ratio_x;
px_size_y= DECIMATED_PIXEL_SIZE(SONY_PIXEL_SIZE) * ratio_y;
gchar*metadata= g_strdup_printf(
"{\"Cur_pos\":{\"x\":%.3f, \"y\":%.3f, \"z\":%.3f}, "
"\"Tar_pos\":{\"x\":%.3f, \"y\":%.3f, \"z\":%.3f}, "
"\"T_base\":{\"x\":%.3f, \"y\":%.3f, \"z\":%.3f}, "
"\"R_base\":[[%.3f, %.3f, %.3f], [%.3f, %.3f, %.3f], [%.3f, %.3f, %.3f]], "
"\"Tilt_offset\":%.3f, "
"\"K\":{\"cx\":%.3f, \"cy\":%.3f, \"fx\":%.3f, \"fy\":%.3f, 
\"px_size_x\":%.3f, \"px_size_y\":%.3f}}",
current_xyz.x, current_xyz.y, current_xyz.z,
target_xyz.x, target_xyz.y, target_xyz.z,
cam_loc.x, cam_loc.y, cam_loc.z,
QTEC_CORE_MATRIX_AT(cam_rot, 0, 0),
QTEC_CORE_MATRIX_AT(cam_rot, 0, 1),
QTEC_CORE_MATRIX_AT(cam_rot, 0, 2),
QTEC_CORE_MATRIX_AT(cam_rot, 1, 0),
QTEC_CORE_MATRIX_AT(cam_rot, 1, 1),
QTEC_CORE_MATRIX_AT(cam_rot, 1, 2),
QTEC_CORE_MATRIX_AT(cam_rot, 2, 0),
QTEC_CORE_MATRIX_AT(cam_rot, 2, 1),
QTEC_CORE_MATRIX_AT(cam_rot, 2, 2),
tof, cx, cy, fx, fy, px_size_x, px_size_y);
gst_tag_setter_add_tags(tagsetter, GST_TAG_MERGE_REPLACE,
GST_TAG_GEO_LOCATION_ELEVATION, f_mm,
GST_TAG_GEO_LOCATION_LATITUDE, pos1.angle,
GST_TAG_GEO_LOCATION_LONGITUDE, pos0.angle,
GST_TAG_ARTIST, "msb at qtec.com",
GST_TAG_DESCRIPTION, metadata,
NULL);
g_free(metadata);
gst_object_unref(mux);
returnGST_PAD_PROBE_OK;
}

As you can see I'm using the GPS (ELEVATION, LATITUDE, LONGITUDE) part 
of EXIF as well as the ARTIST (as author).

And then anything else custom has to go into DESCRIPTION


Best Regards

Marianna

On 08.12.2021 13.00, gstreamer-devel-request at lists.freedesktop.org wrote:
> Hi all,
> i have a node js application that uses ffmpeg to receive an h264 stream
> from a camera and convert to jpeg images.
> the timestamp is overimpressed on the image and is extracted with an OCR
> (tesseract.js)
> My goal is to remove the OCR and extract the NTP timestamp of the frame
> directly.
> I was told gstreamer has all i need but it seems too difficult.
> For now i'm able to build a pipeline to write on file the single jpeg with
> gst launch.
> The problems i see are
> 1)extract the NTP timestamp, 2)synchronize with the jpeg,
> 3)put the timestamp somewhere (maybe metadata?)
> 4)Compile the application to be standalone without requiring to install
> gstreamer
>
> i'm not a developer so i need to know if this is doable for me or i need to
> look elsewhere.
> Any help is much appreciated
> Thanks

-- 
Best regards / Med venlig hilsen
“Marianna Smidth Buschle”
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/gstreamer-devel/attachments/20211210/3e811cdb/attachment-0001.htm>