When trying to debug why certain CoreImage renders increased our CPU usage significantly, I can see in the sample trace that when I crop and composite images in certain ways, CoreImage is doing memcpys. When I take out the crop filter, it goes away. I'm using this for a realtime 2D video pipeline, so this isn't good.
All my images are on the GPU, so I'm stumped as to what it's copying on the CPU.
Anyone have any ideas what this might be doing?
+ ! 457 -[CIContext render:toCVPixelBuffer:bounds:colorSpace:] (in CoreImage) + 1296 [0x7fff8db01111]
+ ! 457 -[CIContext render:toIOSurface:bounds:colorSpace:] (in CoreImage) + 1907 [0x7fff8db030e5]
+ ! 455 CI::image_render_to_surface(CI::Context*, CI::Image*, CGRect, CGColorSpace*, __IOSurface*, CGPoint, CI::PixelFormat) (in CoreImage) + 1862 [0x7fff8db1d8a9]
+ ! : 451 CI::_image_render(char const*, CI::Context*, CI::Image*, CGRect, CGColorSpace*, CI::PixelFormat, unsigned long, CGPoint const&, CI::swizzle_info const&) (in CoreImage) + 869 [0x7fff8db1cdda]
+ ! : | 443 CI::tile_node_graph(CI::Context*, char const*, CI::Node*, CGRect const&, CI::PixelFormat, CI::swizzle_info const&, void (CI::Node*, CGRect) block_pointer) (in CoreImage) + 834 [0x7fff8db1ffe1]
+ ! : | + 416 ___ZN2CIL13_image_renderEPKcPNS_7ContextEPNS_5ImageE6CGRectP12CGColorSpaceNS_11PixelFormatEmRK7CGPointRKNS_12swizzle_infoE_block_invoke (in CoreImage) + 22 [0x7fff8db22d54]
+ ! : | + ! 416 CI::Context::render(CI::Node*, CGRect const&) (in CoreImage) + 75 [0x7fff8db0efb1]
+ ! : | + ! 413 CI::Context::recursive_render(CI::Node*, CGRect const&, CI::Node*) (in CoreImage) + 1285 [0x7fff8db0ee95]
+ ! : | + ! : 277 CI::MetalContext::render_root_node(CI::ProgramNode*, CGRect const&, void () block_pointer) (in CoreImage) + 105 [0x7fff8dc09549]
+ ! : | + ! : | 273 CI::MetalContext::render_node(CI::ProgramNode*, CGRect const&, void const**, unsigned long, bool) (in CoreImage) + 765 [0x7fff8dc08ddb]
+ ! : | + ! : | + 269 CI::MetalContext::bind_arguments(CI::ProgramNode const*, CGRect const&, CGRect const&) (in CoreImage) + 532 [0x7fff8dc0906a]
+ ! : | + ! : | + ! 252 CI::Context::bind_sampler(CI::TextureSampler const*, CGRect const&, int, CI::KernelArgumentType) (in CoreImage) + 992 [0x7fff8db0f3a6]
+ ! : | + ! : | + ! : 232 CI::SurfaceNode::surfaceForROI(CI::Context const*, CGRect) const (in CoreImage) + 718 [0x7fff8db71dc2]
+ ! : | + ! : | + ! : | 142 SurfaceApplyPlaneReadOnlyBlock (in CoreImage) + 170 [0x7fff8dbdcd39]
+ ! : | + ! : | + ! : | + 142 ___ZNK2CI11SurfaceNode13surfaceForROIEPKNS_7ContextE6CGRect_block_invoke (in CoreImage) + 123 [0x7fff8db71e5c]
+ ! : | + ! : | + ! : | + 139 SurfaceApplyPlaneBlock (in CoreImage) + 196 [0x7fff8dbdcbeb]
+ ! : | + ! : | + ! : | + ! 133 ___ZNK2CI11SurfaceNode13surfaceForROIEPKNS_7ContextE6CGRect_block_invoke_2 (in CoreImage) + 367 [0x7fff8db71fd1]
+ ! : | + ! : | + ! : | + ! : 133 _platform_memmove$VARIANT$Haswell (in libsystem_platform.dylib) + 324,224,... [0x7fffa2a63fe4,0x7fffa2a63f80,...]
UPDATE:
Here is the rendering pipeline that includes the crop. This one uses tons of CPU with the memcpy above. All the sources are IOSurfaces. It's taking two images and making a split screen:
<CIImage: 0x618000623e20 extent [infinite]>
colorkernel _sourceOver(src,dst) extent=[infinite]
crop [1280 0 1280 1440] extent=[1280 0 1280 1440]
affine [3.13725 0 0 3.13725 825.098 0] extent=[825.098 0 2189.8 1440]
affine [1 0 0 -1 0 459] extent=[0 0 698 459]
IOSurface 0x610000010300 BGRA8 extent=[0 0 698 459]
colorkernel _sourceOver(src,dst) extent=[infinite]
crop [0 0 1281 1440] extent=[0 0 1281 1440] opaque
affine [-2.00156 -0 0 2 1921.5 0] extent=[-640.5 0 2562 1440] opaque
affine [1 0 0 -1 0 720] extent=[0 0 1280 720] opaque
IOSurface 0x60000000b650 YCC420v 709 alpha_one extent=[0 0 1280 720] opaque
fill [0 0 0 1 "Generic Gray Profile"] extent=[infinite][0 0 1 1] opaque
-
If I take out the crops, this is the pipeline that doesn't use CPU, and doesn't go down the surfaceForROI path:
<CIImage: 0x618000225040 extent [infinite]>
colorkernel _sourceOver(src,dst) extent=[infinite]
affine [3.13725 0 0 3.13725 825.098 0] extent=[825.098 0 2189.8 1440]
affine [1 0 0 -1 0 459] extent=[0 0 698 459]
IOSurface 0x6180000077c0 BGRA8 extent=[0 0 698 459]
colorkernel _sourceOver(src,dst) extent=[infinite]
affine [-2.00156 -0 0 2 1921.5 0] extent=[-640.5 0 2562 1440] opaque
affine [1 0 0 -1 0 720] extent=[0 0 1280 720] opaque
IOSurface 0x61000000c730 YCC420v 709 alpha_one extent=[0 0 1280 720] opaque
fill [0 0 0 1 "Generic Gray Profile"] extent=[infinite][0 0 1 1] opaque
