I'm not sure how it is currently used. Initially the customer was using it to encode the input of a camera transport it to a decoder and display it on a screen. The total encoder+buffer+decoder latency was much less than a frame because they also transported the clock.
This was a few years ago, the design has now been qualified for automotive and it is in prduction as an ASIC.
In general, if you have high bandwidth, the latency can be reduced and, in any case, at least for our encoder, it would make no difference with a JPEG core (from the latency point of view).
Wether it's good enough after decoding for vision algorithms, I'm not sure. Generally H.264 will compress better than JPEG but our core is Baseline, so it's only YUV 4:2:0 8 bits/sample. We have been asked by automotive customers if the sample precision could be increased to 10 or 12 bits (probably for vision algorithms) but never got enough commitment to actually do it.
If you are happy with luminance only at 8 bits/sample, then it should be better than equivalent JPEG. We have also an I frame only version that uses no external DRAM.
Anyway, basic datasheet of the cores are in the download section of our web page :
"H.264 is too lossy and suffers from frame latency as @pmundhenk pointe"
Latency is not an issue for H.264 : our H.264 encoder core has a latency of only 16 video lines (~50 microseconds for 1080p@30) and only because it takes at least 16 video lines to form a 16x16 macroblock, otherwise the real latency for our encoder would be ~3000 clock cycles .
This with P frames and bitrate control. Our core is used in automotive.
Exactly! A carefully adjusted MPEG compression should reach even better results, as higher image quality can be coupled with lower bandwidth. This optimization needs also take into account the end devices that decode the stream. If these have limited processing power, and no additional decoding hardware, information might actually be lost, as the decoding is not fast enough. This is in most cases a minor concern, especially when going to smaller buffers (i.e. less calculation), but one should keep that in mind as well. I think when the overall architecture is well known, one should be able to parametrize MPEG in a way that the delay introduced by the buffer is below 120ms, so not noticeable by humans. This should be a reasonable value for in-vehicle cameras, which are only shown to the user. If these camera images are used for computations, one probably needs to consider an even smaller delay.
And you are right, MJPEG is similar to MPEG without P and I frames. It encodes every frame and can use JPEG decoders which have been heavily optimized for processors. This gives a big advantage, as it does not make sense to replay a delayed frame anyway. Similar to VoIP transmissions.
thank you very much for your offer! I forwarded this also to my colleagues and we are evaluating your portfolio. I will let you know if we need something.
Yes, I agree. Though the speed of the automotive industry is picking up, regarding electronics and software, it is overall still a business with a rather traditional mindset. I see this as a chance for new players in the market though. Tesla has shown what can be done, if you tackle the current problems starting from another perspective. I myself haven't sit in one of their cars, but from images, videos, etc, as well as the data sheets, it looks very impressive. I think the automotive industry needs this push to bring their electronics up and beyong the level of current consumer electronics.
Good point about the delay intrioduced by MPEG, pmundhenk. But even without going to motion JPEG, the MPEG delays are adjustable.
The problem is that in order to get the most possible compression, MPEG likes to create as few full interframes as possible. It introduces interpolated and predictive frames between the big interframes, which of course means you need to intyroduce a long buffer between two interframes. The longer the time, the better the coding efficiency. However, this can be tuned down, reducing the delays and reducing the compression efficiency.
I think your approach of going to MJPEG is essentially the same thing as doing away with the predictive and interplotaed frames of MPEG altogether, and it should still result in at least 10:1 compression. Or thereabouts.
We have had to deal with this issue several times at CogniVue when developing products with our customers. While you are right that the machine vision aspects do not need the high resolution some applications such as Enhanced Backup Camera do still need the "pretty pictures" sent to the infotainment console for display. The result of the vision processing for object detection and distance estimation etc. needs to be overlayed on top of the image displayed.
H.264 is too lossy and suffers from frame latency as @pmundhenk pointed out, MJPEG is still preferred in the Automotive applications. But any amount of compression (ie loss) is problematic if you need to make safety decisions on the camera sensor data. That is why we have our existing CV220X processor family focused on processing close to the sensor and not requiring high bandwidth data transmission across the car chassis. We had to design with the size and power limitations this approach requires.
What are the engineering and design challenges in creating successful IoT devices? These devices are usually small, resource-constrained electronics designed to sense, collect, send, and/or interpret data. Some of the devices need to be smart enough to act upon data in real time, 24/7. Specifically the guests will discuss sensors, security, and lessons from IoT deployments.