Animated webp support by garrettmoon · Pull Request #411 · pinterest/PINRemoteImage

garrettmoon · 2017-09-22T19:52:05Z

This PR switches our technique for GIF handling to keeping in memory frames instead of trying to decode the entire GIF and memory mapping it. That technique was fraught with peril for devices without disk space left and had longer 'startup' costs. In addition there was a pesky race condition I never tracked down.

While I think the former solution has a place, it'll be best used in a hybrid scenario along with the solution introduced here.

Oh, this also adds animated webP support.

ghost · 2017-09-22T19:55:07Z

🚫 CI failed with log

ghost · 2017-09-23T02:11:41Z

🚫 CI failed with log

…imatedImage

ghost · 2017-09-23T02:20:13Z

🚫 CI failed with log

ghost · 2017-09-23T02:37:15Z

🚫 CI failed with log

ghost · 2017-09-23T04:08:49Z

🚫 CI failed with log

garrettmoon · 2017-09-23T17:55:08Z

Ooph, should have updated webp in a different diff. Easiest way to review is search for Source/ those are the important files.

ghost · 2017-09-26T00:40:29Z

🚫 CI failed with log

ghost · 2017-09-27T01:11:35Z

🚫 CI failed with log

ghost · 2017-09-27T14:46:57Z

🚫 CI failed with log

ghost · 2017-09-28T21:01:01Z

🚫 CI failed with log

ghost · 2017-09-29T01:32:16Z

🚫 CI failed with log

ghost · 2017-09-29T03:33:42Z

🚫 CI failed with log

ghost · 2017-09-29T18:00:48Z

🚫 CI failed with log

ghost · 2017-09-30T00:16:07Z

🚫 CI failed with log

PatrickSCLin · 2017-10-02T10:28:01Z

Can't wait...

nguyenhuy

LGTM!

nguyenhuy · 2017-10-04T14:39:36Z

 the same NSData.

- PINAnimatedImage's are also decoded chunks at a time, writing each chunk to a separate file. This allows callback
+ PINGAnimatedImage's are also decoded chunks at a time, writing each chunk to a separate file. This allows callback


Nit: PINMemMapAnimatedImage perhaps?

nguyenhuy · 2017-10-04T14:42:17Z

+{
+    if (_demux) {
+        WebPDemuxDelete(_demux);
+    } if (_durations) {


Nit: new line.

ghost · 2017-10-04T20:17:16Z

	1 Warning
⚠️	This is a big PR, please consider splitting it up to ease code review.

Generated by 🚫 Danger

maicki

LGTM

maicki · 2017-10-05T15:55:48Z

@@ -21,7 +21,7 @@
 static const Float32 maxFileDuration = 1; //max duration of a file in seconds
 static const NSUInteger kCleanupAfterStartupDelay = 10; //clean up files after 10 seconds if it hasn't been done.

-typedef void(^PINAnimatedImageInfoProcessed)(PINImage *coverImage, NSUUID *UUID, Float32 *durations, CFTimeInterval totalDuration, size_t loopCount, size_t frameCount, size_t width, size_t height, size_t bitsPerPixel, UInt32 bitmapInfo);
+typedef void(^PINAnimatedImageInfoProcessed)(PINImage *coverImage, NSUUID *UUID, Float32 *durations, CFTimeInterval totalDuration, size_t loopCount, size_t frameCount, UInt32 width, UInt32 height, size_t bitsPerPixel, UInt32 bitmapInfo);


Any reason we don't use uint32_t in here for width and height too? I saw we cast CGImageSourceGetCount to UInt32, any reason we don't cast that to uint32_t?

Honestly this class is now unused and likely is going to go away so I didn't clean it up. I'm keeping it for now because some of the code might be useful if we decide to make a hybrid memory / disk cache solution.

timonus

All looks good to me, minor feedback.

timonus · 2017-10-10T17:14:24Z

+
+NSString *kPINAnimatedImageErrorDomain = @"kPINAnimatedImageErrorDomain";
+
+const NSTimeInterval kPINAnimatedImageDisplayRefreshRate = 60.0;


This value isn't true on some newer devices (the new iPad Pros). Could this instead be gleaned from UIDevice somehow?

timonus · 2017-10-10T17:15:56Z

+    return MAX(self.minimumFrameInterval * kPINAnimatedImageDisplayRefreshRate, 1);
+}
+
+//Credit to FLAnimatedImage ( https://github.com/Flipboard/FLAnimatedImage ) for display link interval calculations


Not necessary at the moment, but might be good to unit test this piece. Always seemed a bit risky to me!

timonus · 2017-10-10T17:16:07Z

+    return (scaledGCD / kGreatestCommonDivisorPrecision);
+}
+
+//Credit to FLAnimatedImage ( https://github.com/Flipboard/FLAnimatedImage ) for display link interval calculations


Err, perhaps the unit tests should be here actually.

timonus · 2017-10-10T17:20:35Z

+}
+
+@property (atomic, strong) NSDate *lastMemoryWarning;
+@property (atomic, assign) BOOL weAreTheProblem;


I dig the name, but might be worth a comment explaining the meaning of this property ;)

timonus · 2017-10-10T17:29:27Z

+
+                    PINLog(@"Frames left: %ld", (long)_playbackReady);
+
+                    dispatch_block_t notify = nil;


Minor, but this could be moved up a couple levels in scope. It's only referenced in one of the inner ifs here.

timonus · 2017-10-10T17:32:15Z

+    } else {
+        // No caching :(
+        framesToCache = 0;
+    }


Good comments. Might be good in the future to add awareness of multiple live instances of PINCachedAnimatedImage in the future. If you have many live instances in the first bucket above we could still drive the system to run out of memory. You could have each live instance get a memory quota that grows/shrinks with the number of live instances.

timonus · 2017-10-10T17:38:10Z

+                                                   (CFDictionaryRef)@{(__bridge NSString *)kCGImageSourceShouldCache:
+                                                                          (__bridge NSNumber *)kCFBooleanFalse});
+            _width = (uint32_t)[(NSNumber *)imageProperties[(__bridge NSString *)kCGImagePropertyPixelWidth] unsignedIntegerValue];
+            _height = (uint32_t)[(NSNumber *)imageProperties[(__bridge NSString *)kCGImagePropertyPixelHeight] unsignedIntegerValue];


Not a blocker at the moment, but I've seen loading GIF loopCount and frame durations become a bottleneck on larger GIFs. I was working on loading loopCount and the durations asynchronously in timonus/FLAnimatedImage@bddff94 and timonus/FLAnimatedImage@0198b74 respectively, but never finished. Could be a nice thing to have here if we ever notice performance issues during initialization of GIFs. playbackReady could be adjusted to only become true once those are loaded.

timonus · 2017-10-10T17:45:35Z

+- (BOOL)pin_isAnimatedWebP
+{
+    WebPBitstreamFeatures features;
+    if (WebPGetFeatures([self bytes], [self length], &features) == VP8_STATUS_OK) {


Is it safe to call this without checking -pin_isWebP first?

garrettmoon and others added 12 commits September 22, 2017 19:17

Just a test so far

cba7b91

Separating GIF and WebP machinery

9290647

It's alive!

929e0ff

Use iter width and height instead.

d5f3f01

Caching works!

a705c02

Need 2 frames to keep ahead by one.

aef6771

A bunch of TODO'ns

a6ed55e

Ensure CGImageRef isn't released underneath us.

332c15e

Refactored to make it easier to handle memory warnings in PINCachedAn…

dd1ccd3

…imatedImage

Fix up warnings

29860b4

Last todos todone.

ea2cbf1

Update CHANGELOG

ef0f5c9

garrettmoon force-pushed the animatedWebPSupport branch from e89b440 to ef0f5c9 Compare September 23, 2017 02:18

Fix tvOS

5535bb6

Fix macOS

ddbe896

Clean up appleTV project, disable analyzer on webp file.

da17284

Fixes a dumb retain cycle

a10025a

garrettmoon added 2 commits September 25, 2017 19:47

Decode frames serially, fix cleaning up frames

349be19

Now works with non-key frames

2fc02e3

Remove debugging code

ec99a2d

garrettmoon mentioned this pull request Sep 28, 2017

Crash on [PINAnimatedImage frameCount] #389

Closed

Fix test

3225775

GIFs aren't served decoded :/

3a6d33d

Clear out coverImage on cache clear

2db40e4

Fix mac support

daa3fab

Fix analyzer errors

4c5ac08

Fix up analyzer errors

d95e7be

nguyenhuy approved these changes Oct 4, 2017

View reviewed changes

Clean up. Thanks @nguyenhuy!

9bd4ffa

maicki self-requested a review October 5, 2017 15:58

maicki approved these changes Oct 5, 2017

View reviewed changes

garrettmoon merged commit 1d25184 into master Oct 5, 2017

timonus reviewed Oct 10, 2017

View reviewed changes


		NSString *kPINAnimatedImageErrorDomain = @"kPINAnimatedImageErrorDomain";

		const NSTimeInterval kPINAnimatedImageDisplayRefreshRate = 60.0;


		PINLog(@"Frames left: %ld", (long)_playbackReady);

		dispatch_block_t notify = nil;

Uh oh!

Conversation

garrettmoon commented Sep 22, 2017

Uh oh!

ghost commented Sep 22, 2017

Uh oh!

ghost commented Sep 23, 2017

Uh oh!

ghost commented Sep 23, 2017

Uh oh!

ghost commented Sep 23, 2017

Uh oh!

ghost commented Sep 23, 2017

Uh oh!

garrettmoon commented Sep 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Sep 26, 2017

Uh oh!

ghost commented Sep 27, 2017

Uh oh!

ghost commented Sep 27, 2017

Uh oh!

ghost commented Sep 28, 2017

Uh oh!

ghost commented Sep 29, 2017

Uh oh!

ghost commented Sep 29, 2017

Uh oh!

ghost commented Sep 29, 2017

Uh oh!

ghost commented Sep 30, 2017

Uh oh!

PatrickSCLin commented Oct 2, 2017

Uh oh!

nguyenhuy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ghost commented Oct 4, 2017

Uh oh!

maicki left a comment

Choose a reason for hiding this comment

Uh oh!

maicki Oct 5, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timonus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

garrettmoon commented Sep 23, 2017 •

edited

Loading

maicki Oct 5, 2017 •

edited

Loading