RESOLVED FIXED 197384
[CG] Add support for HEIF-sequence (‘public.heics’) images
https://bugs.webkit.org/show_bug.cgi?id=197384
Summary [CG] Add support for HEIF-sequence (‘public.heics’) images
Said Abou-Hallawa
Reported 2019-04-29 13:32:53 PDT
We need to add support for these images in two places: 1) ImageDecoderCG::repetitionCount(): Since HEICS images do not support loopCount, we will be returning RepetitionCountInfinite. 2) ImageDecoderCG::frameDurationAtIndex(): We need to read the values of the following dictionary keys for the frame duration kCGImagePropertyHEICSUnclampedDelayTime and kCGImagePropertyHEICSDelayTime.
Attachments
Patch (3.03 KB, patch)
2019-04-29 13:39 PDT, Said Abou-Hallawa
no flags
Patch (3.82 KB, patch)
2019-04-30 17:27 PDT, Said Abou-Hallawa
no flags
Patch (451.68 KB, patch)
2019-05-03 22:31 PDT, Said Abou-Hallawa
no flags
Patch (451.71 KB, patch)
2019-05-03 23:16 PDT, Said Abou-Hallawa
no flags
Archive of layout-test-results from ews107 for mac-highsierra-wk2 (2.67 MB, application/zip)
2019-05-04 00:37 PDT, EWS Watchlist
no flags
Archive of layout-test-results from ews213 for win-future (14.00 MB, application/zip)
2019-05-04 02:25 PDT, EWS Watchlist
no flags
Patch (452.92 KB, patch)
2019-05-06 09:49 PDT, Said Abou-Hallawa
no flags
Patch (456.18 KB, patch)
2019-05-06 10:48 PDT, Said Abou-Hallawa
no flags
Patch (452.65 KB, patch)
2019-05-06 14:42 PDT, Said Abou-Hallawa
no flags
Archive of layout-test-results from ews213 for win-future (13.68 MB, application/zip)
2019-05-06 16:44 PDT, EWS Watchlist
no flags
Patch (453.71 KB, patch)
2019-05-09 14:59 PDT, Said Abou-Hallawa
no flags
Patch (1000.41 KB, patch)
2019-05-09 18:59 PDT, Said Abou-Hallawa
no flags
Patch (1003.33 KB, patch)
2019-05-10 09:43 PDT, Said Abou-Hallawa
no flags
Patch (1004.88 KB, patch)
2019-05-13 21:41 PDT, Said Abou-Hallawa
no flags
Said Abou-Hallawa
Comment 1 2019-04-29 13:39:27 PDT
Said Abou-Hallawa
Comment 2 2019-04-29 13:40:12 PDT
Said Abou-Hallawa
Comment 3 2019-04-30 17:27:21 PDT
Simon Fraser (smfr)
Comment 4 2019-04-30 17:31:07 PDT
Comment on attachment 368632 [details] Patch Can we add layout tests for this?
Said Abou-Hallawa
Comment 5 2019-05-03 22:31:16 PDT
Said Abou-Hallawa
Comment 6 2019-05-03 23:16:54 PDT
EWS Watchlist
Comment 7 2019-05-04 00:37:40 PDT
Comment on attachment 369059 [details] Patch Attachment 369059 [details] did not pass mac-wk2-ews (mac-wk2): Output: https://webkit-queues.webkit.org/results/12097566 New failing tests: fast/images/animated-heics.html
EWS Watchlist
Comment 8 2019-05-04 00:37:42 PDT
Created attachment 369063 [details] Archive of layout-test-results from ews107 for mac-highsierra-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews107 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
EWS Watchlist
Comment 9 2019-05-04 02:25:56 PDT
Comment on attachment 369059 [details] Patch Attachment 369059 [details] did not pass win-ews (win): Output: https://webkit-queues.webkit.org/results/12098014 New failing tests: svg/repaint/change-background-color.html
EWS Watchlist
Comment 10 2019-05-04 02:25:58 PDT
Created attachment 369067 [details] Archive of layout-test-results from ews213 for win-future The attached test failures were seen while running run-webkit-tests on the win-ews. Bot: ews213 Port: win-future Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Simon Fraser (smfr)
Comment 11 2019-05-04 09:56:30 PDT
Comment on attachment 369059 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369059&action=review r- because I don't think this is correctly resetting state between tests. > Source/WebCore/platform/graphics/cg/UTIRegistry.cpp:81 > void setAdditionalSupportedImageTypes(const Vector<String>& imageTypes) This seem more like appendAdditionSupportedImageTypes, since you can't unregister a type, right? That also implies that you're never cleaning up this state between tests correctly. > Source/WebCore/platform/graphics/cg/UTIRegistry.cpp:94 > +void setAdditionalSupportedImageTypes(const String& imageTypes) If this is only used for testing, maybe append "ForTesting" to the function name.
Said Abou-Hallawa
Comment 12 2019-05-06 09:49:49 PDT
Sam Weinig
Comment 13 2019-05-06 10:22:10 PDT
Comment on attachment 369128 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369128&action=review > Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:50 > +const CFStringRef WebCoreCGImagePropertyHEICSUnclampedDelayTime = CFSTR("UnclampedDelayTime"); This is unused. > Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:367 > + if (CFNumberRef num = (CFNumberRef)CFDictionaryGetValue(heicsProperties, WebCoreCGImagePropertyHEICSDictionary)) Is the property name for the number value really also WebCoreCGImagePropertyHEICSDictionary? It seems like maybe this was supposed to be WebCoreCGImagePropertyHEICSUnclampedDelayTime?
Said Abou-Hallawa
Comment 14 2019-05-06 10:48:20 PDT
Said Abou-Hallawa
Comment 15 2019-05-06 10:57:03 PDT
Comment on attachment 369059 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369059&action=review >> Source/WebCore/platform/graphics/cg/UTIRegistry.cpp:81 >> void setAdditionalSupportedImageTypes(const Vector<String>& imageTypes) > > This seem more like appendAdditionSupportedImageTypes, since you can't unregister a type, right? > > That also implies that you're never cleaning up this state between tests correctly. For DRT and WTR, this function is now called twice. It is called first for the TestOptions then it is called when WebView or WebPage is created. So it is set correctly the first time but it is cleared the second time. I did it this way to prevent clearing it from creating the WebView and the WebPage. But you are right, this fix will not clean the state between tests correctly. I think the solution is to move the condition imageTypes.isEmpty() to WebView and WebPage creation functions. >> Source/WebCore/platform/graphics/cg/UTIRegistry.cpp:94 >> +void setAdditionalSupportedImageTypes(const String& imageTypes) > > If this is only used for testing, maybe append "ForTesting" to the function name. Done.
Said Abou-Hallawa
Comment 16 2019-05-06 10:57:53 PDT
Comment on attachment 369128 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369128&action=review >> Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:367 >> + if (CFNumberRef num = (CFNumberRef)CFDictionaryGetValue(heicsProperties, WebCoreCGImagePropertyHEICSDictionary)) > > Is the property name for the number value really also WebCoreCGImagePropertyHEICSDictionary? It seems like maybe this was supposed to be WebCoreCGImagePropertyHEICSUnclampedDelayTime? Fixed. Thanks for catching this error.
Sam Weinig
Comment 17 2019-05-06 13:02:11 PDT
(In reply to Said Abou-Hallawa from comment #16) > Comment on attachment 369128 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=369128&action=review > > >> Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:367 > >> + if (CFNumberRef num = (CFNumberRef)CFDictionaryGetValue(heicsProperties, WebCoreCGImagePropertyHEICSDictionary)) > > > > Is the property name for the number value really also WebCoreCGImagePropertyHEICSDictionary? It seems like maybe this was supposed to be WebCoreCGImagePropertyHEICSUnclampedDelayTime? > > Fixed. Thanks for catching this error. Please add a test which exercise this path.
Simon Fraser (smfr)
Comment 18 2019-05-06 13:09:31 PDT
Comment on attachment 369136 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369136&action=review > Source/WebKit/WebProcess/WebPage/WebPage.cpp:642 > + if (!parameters.additionalSupportedImageTypes.isEmpty()) > + WebCore::setAdditionalSupportedImageTypes(parameters.additionalSupportedImageTypes); Why the isEmpty check here? Is this making up for the lack of an appendAdditionalSupportedImageTypes? > Source/WebKitLegacy/mac/WebView/WebView.mm:2886 > + if ([[preferences additionalSupportedImageTypes] count]) > + WebCore::setAdditionalSupportedImageTypes(WebCore::webCoreStringVectorFromNSStringArray([preferences additionalSupportedImageTypes])); Why the count check here? > Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.h:173 > bool booleanForKey(WKDictionaryRef, const char* key); > + String stringForKey(WKDictionaryRef, const char* key); These can be static, I think?
Said Abou-Hallawa
Comment 19 2019-05-06 14:42:49 PDT
Said Abou-Hallawa
Comment 20 2019-05-06 15:13:04 PDT
Comment on attachment 369136 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369136&action=review >> Source/WebKit/WebProcess/WebPage/WebPage.cpp:642 >> + WebCore::setAdditionalSupportedImageTypes(parameters.additionalSupportedImageTypes); > > Why the isEmpty check here? Is this making up for the lack of an appendAdditionalSupportedImageTypes? Fixed. The isEmpty() check was removed. I thought the TestOptions will parsed first. Then [WKWebView _initializeWithConfiguration] will be called to create the WTR WKWebView. It turned out I was wrong and [WKWebView _initializeWithConfiguration] is not called in the WTR case. >> Source/WebKitLegacy/mac/WebView/WebView.mm:2886 >> + WebCore::setAdditionalSupportedImageTypes(WebCore::webCoreStringVectorFromNSStringArray([preferences additionalSupportedImageTypes])); > > Why the count check here? Fixed. the count check was removed. >> Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.h:173 >> + String stringForKey(WKDictionaryRef, const char* key); > > These can be static, I think? No. They can't be static nor can they even be const. The reason is these functions call InjectedBundle::outputText() which is not static or const.
Said Abou-Hallawa
Comment 21 2019-05-06 15:14:46 PDT
(In reply to Sam Weinig from comment #17) > (In reply to Said Abou-Hallawa from comment #16) > > Comment on attachment 369128 [details] > > Patch > > > > View in context: > > https://bugs.webkit.org/attachment.cgi?id=369128&action=review > > > > >> Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:367 > > >> + if (CFNumberRef num = (CFNumberRef)CFDictionaryGetValue(heicsProperties, WebCoreCGImagePropertyHEICSDictionary)) > > > > > > Is the property name for the number value really also WebCoreCGImagePropertyHEICSDictionary? It seems like maybe this was supposed to be WebCoreCGImagePropertyHEICSUnclampedDelayTime? > > > > Fixed. Thanks for catching this error. > > Please add a test which exercise this path. I do not know any tool which can edit/create a HEICS image.
EWS Watchlist
Comment 22 2019-05-06 16:44:31 PDT
Comment on attachment 369169 [details] Patch Attachment 369169 [details] did not pass win-ews (win): Output: https://webkit-queues.webkit.org/results/12116421 New failing tests: legacy-animation-engine/compositing/reflections/load-video-in-reflection.html
EWS Watchlist
Comment 23 2019-05-06 16:44:34 PDT
Created attachment 369205 [details] Archive of layout-test-results from ews213 for win-future The attached test failures were seen while running run-webkit-tests on the win-ews. Bot: ews213 Port: win-future Platform: CYGWIN_NT-10.0-17763-3.0.5-338.x86_64-x86_64-64bit
Sam Weinig
Comment 24 2019-05-06 19:00:16 PDT
(In reply to Said Abou-Hallawa from comment #21) > (In reply to Sam Weinig from comment #17) > > (In reply to Said Abou-Hallawa from comment #16) > > > Comment on attachment 369128 [details] > > > Patch > > > > > > View in context: > > > https://bugs.webkit.org/attachment.cgi?id=369128&action=review > > > > > > >> Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:367 > > > >> + if (CFNumberRef num = (CFNumberRef)CFDictionaryGetValue(heicsProperties, WebCoreCGImagePropertyHEICSDictionary)) > > > > > > > > Is the property name for the number value really also WebCoreCGImagePropertyHEICSDictionary? It seems like maybe this was supposed to be WebCoreCGImagePropertyHEICSUnclampedDelayTime? > > > > > > Fixed. Thanks for catching this error. > > > > Please add a test which exercise this path. > > I do not know any tool which can edit/create a HEICS image. Can you ask the ImageIO folks for one?
Said Abou-Hallawa
Comment 25 2019-05-09 14:59:58 PDT
Said Abou-Hallawa
Comment 26 2019-05-09 18:59:30 PDT
Said Abou-Hallawa
Comment 27 2019-05-10 09:43:10 PDT
Said Abou-Hallawa
Comment 28 2019-05-10 09:44:50 PDT
I cleaned ImageDecoderCG::repetitionCount() and ImageDecoderCG::frameDurationAtIndex() by deleting the repeating code and avoiding unnecessary calls to the image properties dictionary.
Simon Fraser (smfr)
Comment 29 2019-05-13 14:32:31 PDT
Comment on attachment 369556 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=369556&action=review > Source/WebCore/platform/graphics/cg/ImageDecoderCG.cpp:340 > + if (properties) > + frameProperties = (CFDictionaryRef)CFDictionaryGetValue(properties.get(), kCGImagePropertyGIFDictionary); > + > + if (properties && !frameProperties) > + frameProperties = (CFDictionaryRef)CFDictionaryGetValue(properties.get(), kCGImagePropertyPNGDictionary); > + This looks like the code above. Maybe share into a small static function.
Said Abou-Hallawa
Comment 30 2019-05-13 21:41:11 PDT
WebKit Commit Bot
Comment 31 2019-05-14 08:14:03 PDT
Comment on attachment 369819 [details] Patch Clearing flags on attachment: 369819 Committed r245280: <https://trac.webkit.org/changeset/245280>
WebKit Commit Bot
Comment 32 2019-05-14 08:14:05 PDT
All reviewed patches have been landed. Closing bug.
Said Abou-Hallawa
Comment 33 2019-05-14 11:28:22 PDT
The tests, which were submitted with this patch, have to be skipped in WebKit for now. Committed r245292: <https://trac.webkit.org/changeset/245292>
Note You need to log in before you can comment on or make changes to this bug.