前面两篇文章介绍了图像生产者和图像消费者,终于到了最后一篇——图像缓冲区,这三者一起构成了一个完整的Android图像显示系统。Android中的图像生产者OpenGL,Skia,Vulkan将绘制的数据存放在图像缓冲区中,Android中的图像消费SurfaceFlinger从图像缓冲区将数据取出,进行加工及合成。那么图像缓冲区是什么呢?它是如何创建出来的呢?又要如何使用它呢?它的存储原理是什么呢?读完这篇文章,你就能回答这些问题了。
在讲解图像的生产者时,多次提到了Surface,我们知道Surface可以存储用来绘制的图形数据。在硬件加速中,需要调用 ThreadedRenderer.initialize(mSurface) 函数将Surface缓冲区传递到OpenGL或者Vulkan的渲染管线;在软件绘制中调用 drawSoftware(surface, mAttachInfo, xOffset, yOffset, scalingRequired, dirty) 函数将Surface缓冲区传递到native层的SkiaCanvas。当OpenGL,Vulkan或Skia拥有Surface后,就可以获取到图形缓冲区,将需要绘制的内容存储在图形缓冲区内了。
同时,在讲解图像的消费者时,也多次提到了Layer,我们知道通过Layer可以取出缓冲区的图形数据。在SurfaceFlinger需要消费图像数据,进行图层混合时,需要遍历Layer,取出Layer中缓冲区的图形数据后,再加过或合成处理。
我们也知道一个Layer唯一对应了一个Surface,他们的关系如下。
所以,Surface和Layer都属于图形缓冲区的组成部分。那么Android图形缓冲区由哪些部分组成呢?Surface和Layer又是什么呢?他们是怎么关联起来的呢?他们又分别是如何创建、获取以及使用缓冲区的呢?这篇文章主要的内容都是围绕这四个问题来讲解,在深入讲解这四个问题之前,这里我们先高屋建瓴的了解这个四个问题的答案,这样在接下面的篇幅中,才不会迷失在冗长的代码中。
1.Android图形缓冲区由哪些部分组成呢?
Android的图形缓冲区由Surface,BufferQueue,Layer,GraphicBuffer四部分组成。BufferQueue中的slots数组最多能存储64个GraphicBuffer,而GraphicBuffer就是真正被分配内存,并能存储图形数据的缓冲区2.Surface和Layer又是什么呢?
Surface是提供给图形生产者控制缓冲区的类,Surface持有GraphicBufferProducer,简称gbp,gbp专门用来创建或获取可用的GraphicBuffer以及提交绘制后的GraphicBufferLayer是提供给图像消费者获取缓冲区的类,Layer持有GraphicBufferConsumer,简称gbc,通过gbc用来获取和释放GraphicBuffer3.他们是怎么关联起来的呢?
Surface和Layer通过BufferQueue关联起来,Surface持有BufferQueue中的gbp,Layer持有BufferQueue中的gbc,gbp和gbc的GraphicBuffer都存储在GraphicBufferCore的slots数组中。4.如何创建、获取以及使用缓冲区呢?
Surface通过调用gbp的dequeue函数获取GraphicBuffer,调用queue函数提交使用完毕的GraphicBuffer。Layer通过调用gbc的acquire函数获取有数据的GraphicBuffer,调用release释放GraphicBuffer总结一下上面提到的知识:当我们想要绘制图像时,需要创建Surface和Layer,图像生产者如Skia,OpenGL通过Surface调用dequeue函数获取一块缓冲区GraphicBuffer,有了这块缓冲区,就可以在缓冲区上绘制图像了,当绘制完毕后,通过Surface调用queue函数,将GraphicBuffer归还到BufferQueue。之后,Vsync通知图像消费者SurfaceFlinger调用Layer的acquire函数,获取绘制好内容的GraphicBuffer进行合成与处理,处理完毕后通过release函数释放这块缓冲区。
了解了大概的背景和流程,我们接着开始对图像缓冲区的深入学习。
我们已经知道缓冲区由Surface,Layer,BufferQueue和GraphicBuffer组成。这一节主要讲这四个部分是如何创建的,先从Surface和Layer的创建开始。
为了更容易的理解如何创建Surface和Layer,这里先从开机动画这个案例讲起。
在前面图像生产者中讲如何通过OpenGL ES播放开启动画时,已经提到了创建Surface的流程,它的流程很简单,这里再回顾一下通过Android开机动画的启动流程。开机动画的对象为BootAnimation,它的构造函数如下。
/frameworks/base/cmds/bootanimation/BootAnimation.cpp
BootAnimation::BootAnimation() : Thread(false), mClockEnabled(true), mTimeIsAccurate(false), mTimeFormat12Hour(false), mTimeCheckThread(NULL) { //创建SurfaceComposerClient mSession = new SurfaceComposerClient(); …… }可以看到构造函数里面创建了SurfaceComposerClient。
我们接着看BootAnimation的初始化函数——readyToRun函数
/frameworks/base/cmds/bootanimation/BootAnimation.cpp
status_t BootAnimation::readyToRun() { mAssets.addDefaultAssets(); sp<IBinder> dtoken(SurfaceComposerClient::getBuiltInDisplay( ISurfaceComposer::eDisplayIdMain)); DisplayInfo dinfo; //获取屏幕信息 status_t status = SurfaceComposerClient::getDisplayInfo(dtoken, &dinfo); if (status) return -1; // 通知SurfaceFlinger创建Surface,创建成功会返回一个SurfaceControl代理 sp<SurfaceControl> control = session()->createSurface(String8("BootAnimation"), dinfo.w, dinfo.h, PIXEL_FORMAT_RGB_565); SurfaceComposerClient::openGlobalTransaction(); //设置这个layer在SurfaceFlinger中的层级顺序 control->setLayer(0x40000000); //获取surface sp<Surface> s = control->getSurface(); // 以下是EGL的初始化流程 const EGLint attribs[] = { EGL_RED_SIZE, 8, EGL_GREEN_SIZE, 8, EGL_BLUE_SIZE, 8, EGL_DEPTH_SIZE, 0, EGL_NONE }; EGLint w, h; EGLint numConfigs; EGLConfig config; EGLSurface surface; EGLContext context; //步骤1:获取Display EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY); //步骤2:初始化EGL eglInitialize(display, 0, 0); //步骤3:选择参数 eglChooseConfig(display, attribs, &config, 1, &numConfigs); //步骤4:传入SurfaceFlinger生成的surface,并以此构造EGLSurface surface = eglCreateWindowSurface(display, config, s.get(), NULL); //步骤5:构造egl上下文 context = eglCreateContext(display, config, NULL, NULL); //步骤6:绑定EGL上下文 if (eglMakeCurrent(display, surface, surface, context) == EGL_FALSE) return NO_INIT; …… }通过BootAnimation的构造函数和readyToRun函数可以看到,创建Surface的步骤如下:
创建SurfaceComponentClient
通过SurfaceComponentClient的createSurface函数创建SurfaceControl,SurfaceControl是Surface的控制类
有了SurfaceControl之后,我们就可以并能通过getSurface获取到Surface
只需要三步,Surface的创建就完成了,非常的简单。但是不我们不能止步于如何创建Surface,我们还需要了解SurfaceComponentClient什么?createSurface经历了哪些流程?getSurface又经历了哪些流程?
我们先看看SurfaceComponentClient对象,它的构造函数如下。
/frameworks/native/libs/gui/SurfaceComposerClient.cpp
ComposerService::ComposerService() : Singleton<ComposerService>() { connectLocked(); } void ComposerService::connectLocked() { const String16 name("SurfaceFlinger"); //获取SurfaceFlinger while (getService(name, &mComposerService) != NO_ERROR) { usleep(250000); } //注册binder death的通知 …… }从SurfaceComposerClient可以看到,它在connectLocked函数中获取了SurfaceFlinger的客户端Binder代理mComposerService。
接着在看看它初始化函数onFirstRef
/frameworks/native/libs/gui/SurfaceComposerClient.cpp
void SurfaceComposerClient::onFirstRef() { sp<ISurfaceComposer> sm(ComposerService::getComposerService()); if (sm != 0) { auto rootProducer = mParent.promote(); sp<ISurfaceComposerClient> conn; conn = (rootProducer != nullptr) ? sm->createScopedConnection(rootProducer) : sm->createConnection(); if (conn != 0) { mClient = conn; mStatus = NO_ERROR; } } }onFirstRef函数中通过mComposerService调用createConnection方法。mComposerService就是SurfaceFlinger的binder代理,所以这里会最终调用SurfaceFlinger的createConnection函数。
接着看SurfaceFlinger的createConnection函数
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() { return initClient(new Client(this)); } static sp<ISurfaceComposerClient> initClient(const sp<Client>& client) { status_t err = client->initCheck(); if (err == NO_ERROR) { return client; } return nullptr; }可以看到createConnection方法创建了Client,这个Client封装了对Layer和Surface的操作,我们看一下Client的头文件
/frameworks/native/services/surfaceflinger/Client.h
class Client : public BnSurfaceComposerClient { public: explicit Client(const sp<SurfaceFlinger>& flinger); Client(const sp<SurfaceFlinger>& flinger, const sp<Layer>& parentLayer); ~Client(); status_t initCheck() const; // protected by SurfaceFlinger::mStateLock void attachLayer(const sp<IBinder>& handle, const sp<Layer>& layer); void detachLayer(const Layer* layer); sp<Layer> getLayerUser(const sp<IBinder>& handle) const; void setParentLayer(const sp<Layer>& parentLayer); private: // ISurfaceComposerClient interface virtual status_t createSurface( const String8& name, uint32_t w, uint32_t h,PixelFormat format, uint32_t flags, const sp<IBinder>& parent, uint32_t windowType, uint32_t ownerUid, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp); virtual status_t destroySurface(const sp<IBinder>& handle); virtual status_t clearLayerFrameStats(const sp<IBinder>& handle) const; virtual status_t getLayerFrameStats(const sp<IBinder>& handle, FrameStats* outStats) const; virtual status_t onTransact( uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags); sp<Layer> getParentLayer(bool* outParentDied = nullptr) const; // constant sp<SurfaceFlinger> mFlinger; // protected by mLock DefaultKeyedVector< wp<IBinder>, wp<Layer> > mLayers; wp<Layer> mParentLayer; // thread-safe mutable Mutex mLock; }; }可以看到,Client继承自BnSurfaceComposerClient,并提供了创建和销毁Layer和Surface的操作函数。
创建好了SurfaceComponentClinet,并有了一个在SurfaceFlinger对应的Clinet。我们可以接着看第二步:createSurface函数。
/frameworks/native/libs/gui/SurfaceComposerClient.cpp
sp<SurfaceControl> SurfaceComposerClient::createSurface( const String8& name, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags, SurfaceControl* parent, uint32_t windowType, uint32_t ownerUid) { sp<SurfaceControl> sur; if (mStatus == NO_ERROR) { sp<IBinder> handle; sp<IBinder> parentHandle; sp<IGraphicBufferProducer> gbp; if (parent != nullptr) { parentHandle = parent->getHandle(); } status_t err = mClient->createSurface(name, w, h, format, flags, parentHandle, windowType, ownerUid, &handle, &gbp); if (err == NO_ERROR) { sur = new SurfaceControl(this, handle, gbp); } } return sur; }createSurface函数主要做了两件事情
调用Client的createSurface方法创建SurfaceControl先看第一件事情:调用Client的createSurface函数
/frameworks/native/services/surfaceflinger/Client.cpp
status_t Client::createSurface( const String8& name, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags, const sp<IBinder>& parentHandle, uint32_t windowType, uint32_t ownerUid, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp) { …… class MessageCreateLayer : public MessageBase { SurfaceFlinger* flinger; Client* client; sp<IBinder>* handle; sp<IGraphicBufferProducer>* gbp; status_t result; const String8& name; uint32_t w, h; PixelFormat format; uint32_t flags; sp<Layer>* parent; uint32_t windowType; uint32_t ownerUid; public: MessageCreateLayer(SurfaceFlinger* flinger, const String8& name, Client* client, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags, sp<IBinder>* handle, uint32_t windowType, uint32_t ownerUid, sp<IGraphicBufferProducer>* gbp, sp<Layer>* parent) : flinger(flinger), client(client), handle(handle), gbp(gbp), result(NO_ERROR), name(name), w(w), h(h), format(format), flags(flags), parent(parent), windowType(windowType), ownerUid(ownerUid) { } status_t getResult() const { return result; } virtual bool handler() { result = flinger->createLayer(name, client, w, h, format, flags, windowType, ownerUid, handle, gbp, parent); return true; } }; sp<MessageBase> msg = new MessageCreateLayer(mFlinger.get(), name, this, w, h, format, flags, handle, windowType, ownerUid, gbp, &parent); mFlinger->postMessageSync(msg); return static_cast<MessageCreateLayer*>( msg.get() )->getResult(); }可以看到,createSurface实际是通过SurfaceFlinger的消息队列调用createLayer函数,接着看SurfaceFlinger中createLayer函数的实现。
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
status_t SurfaceFlinger::createLayer( const String8& name, const sp<Client>& client, uint32_t w, uint32_t h, PixelFormat format, uint32_t flags, uint32_t windowType, uint32_t ownerUid, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp, sp<Layer>* parent) { status_t result = NO_ERROR; sp<Layer> layer; String8 uniqueName = getUniqueLayerName(name); switch (flags & ISurfaceComposerClient::eFXSurfaceMask) { case ISurfaceComposerClient::eFXSurfaceNormal: //创建普通的Layer result = createNormalLayer(client, uniqueName, w, h, flags, format, handle, gbp, &layer); break; case ISurfaceComposerClient::eFXSurfaceDim: //创建有遮罩效果的Layer result = createDimLayer(client, uniqueName, w, h, flags, handle, gbp, &layer); break; default: result = BAD_VALUE; break; } if (result != NO_ERROR) { return result; } layer->setInfo(windowType, ownerUid); result = addClientLayer(client, *handle, *gbp, layer, *parent); if (result != NO_ERROR) { return result; } mInterceptor.saveSurfaceCreation(layer); setTransactionFlags(eTransactionNeeded); return result; } status_t SurfaceFlinger::createNormalLayer(const sp<Client>& client, const String8& name, uint32_t w, uint32_t h, uint32_t flags, PixelFormat& format, sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp, sp<Layer>* outLayer) { // initialize the surfaces switch (format) { case PIXEL_FORMAT_TRANSPARENT: case PIXEL_FORMAT_TRANSLUCENT: format = PIXEL_FORMAT_RGBA_8888; break; case PIXEL_FORMAT_OPAQUE: format = PIXEL_FORMAT_RGBX_8888; break; } *outLayer = new Layer(this, client, name, w, h, flags); status_t err = (*outLayer)->setBuffers(w, h, format, flags); if (err == NO_ERROR) { *handle = (*outLayer)->getHandle(); *gbp = (*outLayer)->getProducer(); } ALOGE_IF(err, "createNormalLayer() failed (%s)", strerror(-err)); return err; }到这里我们可以发现,原来createSurface函数实际上并不是创建Surface,而是创建了Layer。接着看Layer的初始化函数onFirstRef
/frameworks/native/services/surfaceflinger/Layer.cpp
void Layer::onFirstRef() { sp<IGraphicBufferProducer> producer; sp<IGraphicBufferConsumer> consumer; //创建BufferQueue BufferQueue::createBufferQueue(&producer, &consumer, true); mProducer = new MonitoredProducer(producer, mFlinger, this); mSurfaceFlingerConsumer = new SurfaceFlingerConsumer(consumer, mTextureName, this); mSurfaceFlingerConsumer->setConsumerUsageBits(getEffectiveUsage(0)); mSurfaceFlingerConsumer->setContentsChangedListener(this); mSurfaceFlingerConsumer->setName(mName); if (mFlinger->isLayerTripleBufferingDisabled()) { mProducer->setMaxDequeuedBufferCount(2); } const sp<const DisplayDevice> hw(mFlinger->getDefaultDisplayDevice()); updateTransformHint(hw); }可以看到,Layer的初始化函数里通过BufferQueue::createBufferQueue创建了BufferQueue,而BufferQueue又会创建GraphicBufferProducer和GraphicBufferConsumer,关于BufferQueue后面在详讲。
在看第二件事情:创建SurfaceControl
/frameworks/native/libs/gui/SurfaceControl.cpp
SurfaceControl::SurfaceControl( const sp<SurfaceComposerClient>& client, const sp<IBinder>& handle, const sp<IGraphicBufferProducer>& gbp) : mClient(client), mHandle(handle), mGraphicBufferProducer(gbp) { }SurfaceControl的构造函数没有任何操作,仅仅只是传入Client,以及在BufferQueue中创建的GraphicBufferProducer。
Layer和SurfaceControl都创建好了,就差最后一步了:调用getSurface函数获取Surface。
/frameworks/native/libs/gui/SurfaceControl.cpp
sp<Surface> SurfaceControl::getSurface() const { Mutex::Autolock _l(mLock); if (mSurfaceData == 0) { return generateSurfaceLocked(); } return mSurfaceData; } sp<Surface> SurfaceControl::generateSurfaceLocked() const { mSurfaceData = new Surface(mGraphicBufferProducer, false); return mSurfaceData; }可以看到,这里会创建一个Surface。为了对Surface有一个了解,我们看一下Surface的头文件
/frameworks/native/include/gui/Surface.h
class Surface : public ANativeObjectBase<ANativeWindow, Surface, RefBase> { public: explicit Surface(const sp<IGraphicBufferProducer>& bufferProducer, bool controlledByApp = false); …… protected: virtual int dequeueBuffer(ANativeWindowBuffer** buffer, int* fenceFd); virtual int queueBuffer(ANativeWindowBuffer* buffer, int fenceFd); …… protected: uint32_t mReqWidth; uint32_t mReqHeight; PixelFormat mReqFormat; uint32_t mDefaultWidth; uint32_t mDefaultHeight; …… }从头文件可以看到,Surface继承自ANativeWindow,并且封装了许多对Buffer进行操作的方法,以及维护了Surface的大小即属性等参数数据。到这里,Surface也创建完成了,Surface的构造函数的入参就是GraphicBufferProducer。有了Surface,就可以交给OpenGL进行开机动画的绘制了,绘制的过程在图像生产者中已经讲了,这里就不再说了。
通过开机动画这个案例,我们可以发现,通过简单三步,就能创建好Surface和Layer了。下面接着来看看位于Java层的Activity是如何创建Surface,由于Activity的的界面显示流程涉及到很多JNI的通信以及对界面的测量布局等流程,所以它创建Surface和Layer并没有开机动画这么直观易懂。
Activity创建Surface和Layer的流程会和界面绘制、AMS对界面Z轴排序等流程耦合在一块,所以在讲Surface和Layer的创建过程中,会将这些流程一起讲,Activity的界面我都以Window,即窗口来表示。
先看看Java层的Surface,它位于ViewRootImpl中,并且声明成员变量就创建好了。
/frameworks/base/core/java/android/view/ViewRootImpl.java
public final class ViewRootImpl implements ViewParent,View.AttachInfo.Callbacks, ThreadedRenderer.DrawCallbacks { //…… final Surface mSurface = new Surface(); //…… }这里的Surface只是Java层的Surface,它需要绑定Native的Surface,我们一般在构造函数中,可以看到jni方法创建native层对应的对象,比如图像生产者中提到的DisplayListCanvas就会在构造函数中创建native层的RecordingCanvas,所以我们先看看Surface的构造函数
/frameworks/base/core/java/android/view/Surface.java
public Surface() { }这里发现他是一个空实现,找不到创建创建native层Surface的JNI方法调用。那么native层的Surface是如何创建的呢?又是如何和java层的Surface绑定起来的呢?带着这个疑问,接着往下看。
这需要从Activity界面的绘制流程开始讲起,当我们在Activity的OnCreate函数中通过setContentView设置界面后,最终会执行到ViewRootImpl的setView方法,我们从这儿开始看起。
/frameworks/base/core/java/android/view/ViewRootImpl.java
public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView) { synchronized (this) { if (mView == null) { mView = view; //…… // 开启硬件加速 if (mSurfaceHolder == null) { enableHardwareAcceleration(attrs); } //1,测量,布局和绘制流程 requestLayout(); if ((mWindowAttributes.inputFeatures & WindowManager.LayoutParams.INPUT_FEATURE_NO_INPUT_CHANNEL) == 0) { //触摸事件回调信道 mInputChannel = new InputChannel(); } mForceDecorViewVisibility = (mWindowAttributes.privateFlags & PRIVATE_FLAG_FORCE_DECOR_VIEW_VISIBILITY) != 0; mOrigWindowType = mWindowAttributes.type; mAttachInfo.mRecomputeGlobalAttributes = true; collectViewAttributes(); //2,调用wms的addToDisplay函数 res = mWindowSession.addToDisplay(mWindow, mSeq, mWindowAttributes, getHostVisibility(), mDisplay.getDisplayId(), mAttachInfo.mContentInsets, mAttachInfo.mStableInsets, mAttachInfo.mOutsets, mInputChannel); //添加窗口错误的处理 …… //触摸事件监听初始化处理 …… } } }setView函数中关键流程主要是这两件事情:
1,执行requestLayout函数,这个流程里会创建Surface和Layer
2,调用wms的addTodisplay(),这个流程里会创建SurfaceComponentClient
先看看requestLayout函数
/frameworks/base/core/java/android/view/ViewRootImpl.java
public void requestLayout() { if (!mHandlingLayoutInLayoutRequest) { checkThread(); mLayoutRequested = true; scheduleTraversals(); } } void scheduleTraversals() { if (!mTraversalScheduled) { mTraversalScheduled = true; mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier(); mChoreographer.postCallback( Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null); if (!mUnbufferedInputDispatch) { scheduleConsumeBatchedInput(); } notifyRendererOfFramePending(); pokeDrawLockIfNeeded(); } }requestLayout函数通过mChoreographer添加了回调对象mTraversalRunnable,mChoreographer会在收到VSync执行mTraversalRunnable,这里就不介绍mChoreographer了。mTraversalRunnable里做的事情主要是测量,布局和绘制流程,但是现在就进行这些工作还太早了,因为我们的SurfaceComponentClient还没创建好,所以mTraversalRunnable的回调会在addTodisplay后执行。
我们先看mWindowSession.addToDisplay()的流程,他会在mTraversalRunnable回调之前执行。在这这里我们先了解一下mWindowSession,凡是带Session的类都是用来通信的,WindowSession就是Java层的窗口和WMS通信的会话对象,它在ViewRootImp的l构造函数中通过WindowManagerGlobal.getWindowSession()获取。
/frameworks/base/core/java/android/view/ViewRootImpl.java
public ViewRootImpl(Context context, Display display) { …… mWindowSession = WindowManagerGlobal.getWindowSession(); …… } public final class WindowManagerGlobal { …… public static IWindowSession getWindowSession() { synchronized (WindowManagerGlobal.class) { if (sWindowSession == null) { try { InputMethodManager imm = InputMethodManager.getInstance(); IWindowManager windowManager = getWindowManagerService(); sWindowSession = windowManager.openSession( new IWindowSessionCallback.Stub() { @Override public void onAnimatorScaleChanged(float scale) { ValueAnimator.setDurationScale(scale); } }, imm.getClient(), imm.getInputContext()); } catch (RemoteException e) { throw e.rethrowFromSystemServer(); } } return sWindowSession; } } …… }可以看到getWindowSession函数中获取了WindowManagerService的Binder代理,然后调用WindowManagerService的openSession方法来创建Session。
/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java
public IWindowSession openSession(IWindowSessionCallback callback, IInputMethodClient client, IInputContext inputContext) { if (client == null) throw new IllegalArgumentException("null client"); if (inputContext == null) throw new IllegalArgumentException("null inputContext"); Session session = new Session(this, callback, client, inputContext); return session; }为什么不通过WMS的Proxy直接进行通信呢,而要创建Session来进行通信呢?我的理解是WMS要和多个窗口通信,如果直接通过WMS对话会不方便维护对应窗口的上下文,这个Session作用是表示当前这个窗口和WMS的对话。继续回到mWindowSession.addToDisplay函数上来。
/frameworks/base/services/core/java/com/android/server/wm/Session.java
public int addToDisplay(IWindow window, int seq, WindowManager.LayoutParams attrs, int viewVisibility, int displayId, Rect outContentInsets, Rect outStableInsets, Rect outOutsets, InputChannel outInputChannel) { return mService.addWindow(this, window, seq, attrs, viewVisibility, displayId, outContentInsets, outStableInsets, outOutsets, outInputChannel); }addToDisplay方法只是调用了WMS的addWindow方法
/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java
public int addWindow(Session session, IWindow client, int seq, WindowManager.LayoutParams attrs, int viewVisibility, int displayId, Rect outContentInsets, Rect outStableInsets, Rect outOutsets, InputChannel outInputChannel) { int[] appOp = new int[1]; //权限检查 int res = mPolicy.checkAddPermission(attrs, appOp); if (res != WindowManagerGlobal.ADD_OKAY) { return res; } boolean reportNewConfig = false; WindowState parentWindow = null; long origId; final int callingUid = Binder.getCallingUid(); final int type = attrs.type; synchronized(mWindowMap) { if (!mDisplayReady) { throw new IllegalStateException("Display has not been initialialized"); } //创建或获取DisplayContent final DisplayContent displayContent = mRoot.getDisplayContentOrCreate(displayId); if (displayContent == null) { Slog.w(TAG_WM, "Attempted to add window to a display that does not exist: " + displayId + ". Aborting."); return WindowManagerGlobal.ADD_INVALID_DISPLAY; } //窗口类型校验 …… //获取WindowToken AppWindowToken atoken = null; final boolean hasParent = parentWindow != null; WindowToken token = displayContent.getWindowToken( hasParent ? parentWindow.mAttrs.token : attrs.token); final int rootType = hasParent ? parentWindow.mAttrs.type : type; boolean addToastWindowRequiresToken = false; if (token == null) { // token为空的情况下,根据窗口类型判断是否返回错误,有些窗口,如系统窗口运行token为空,但是子窗口等窗口不允许 …… //为允许token为空的窗口创建token final IBinder binder = attrs.token != null ? attrs.token : client.asBinder(); token = new WindowToken(this, binder, type, false, displayContent, session.mCanAddInternalSystemWindow); } else if (rootType >= FIRST_APPLICATION_WINDOW && rootType <= LAST_APPLICATION_WINDOW) { // APPLICATION类型的窗口检查,对应的WindowToken的类型也为APPLICATION …… } else if (rootType == TYPE_INPUT_METHOD) { // INPUT_METHOD类型的窗口检查 …… } else if (rootType == TYPE_VOICE_INTERACTION) { // VOICE类型的窗口检查 …… } else if (rootType == TYPE_WALLPAPER) { // WALLPAPER类型的窗口检查 …… } else if (rootType == TYPE_DREAM) { // TYPE_DREAM类型的窗口检查 …… } else if (rootType == TYPE_ACCESSIBILITY_OVERLAY) { …… } else if (type == TYPE_TOAST) { …… } else if (type == TYPE_QS_DIALOG) { …… } else if (token.asAppWindowToken() != null) { …… } //为这个窗口创建WindowState final WindowState win = new WindowState(this, session, client, token, parentWindow, appOp[0], seq, attrs, viewVisibility, session.mUid, session.mCanAddInternalSystemWindow); …… //执行attach win.attach(); //将window存入mWindowMap mWindowMap.put(client.asBinder(), win); final AppWindowToken aToken = token.asAppWindowToken(); //同一个token会有多个window win.mToken.addWindow(win); …… } return res; }addToDisplay比较长,但主要做的事情只有这几件
对添加窗口的Token和类型合法性校验,Token是这个窗口的标识,Activity窗口的Token会在AMS创建Acitvity的时候创建,系统窗口会在这里创建。创建WindowState,WindowState会持有在上面创建的用于WMS和窗口通信的Session以及一些窗口状态。执行attach函数,这个函数中会创建SurfaceComponentClient存储WindowState这里只关心SurfaceComponentClient的创建,所以我们只看win.attach()函数
/frameworks/base/services/core/java/com/android/server/wm/WindowState.java
void attach() { mSession.windowAddedLocked(mAttrs.packageName); }这里调用了Session的windowAddedLocked函数,这个session就是我们在上面创建的Session,
/frameworks/base/services/core/java/com/android/server/wm/Session.java
void windowAddedLocked(String packageName) { mPackageName = packageName; if (mSurfaceSession == null) { mSurfaceSession = new SurfaceSession(); mService.mSessions.add(this); if (mLastReportedAnimatorScale != mService.getCurrentAnimatorScale()) { mService.dispatchNewAnimatorScaleLocked(this); } } mNumWindow++; }可以看到这里创建了SurfaceSession,又遇到了一个以Session命名的类,它是和谁通信用的呢?带着疑问接着往下看
/frameworks/base/core/java/android/view/SurfaceSession.java
public SurfaceSession() { mNativeClient = nativeCreate(); }SurfaceSession的构造函数调用了native函数nativeCreate
/frameworks/base/core/jni/android_view_SurfaceSession.cpp
static jlong nativeCreate(JNIEnv* env, jclass clazz) { SurfaceComposerClient* client = new SurfaceComposerClient(); client->incStrong((void*)nativeCreate); return reinterpret_cast<jlong>(client); }到这里终于发现了SurfaceComposerClient的身影,在前面已经知道,只要有了SurfaceComposerClient,就能通过createSurface和getSurface创建Layer和Surface了。
我们接着探索Activity的界面显示流程中是在哪儿调用这两个函数的。
那么createSurface的调用在哪儿呢?它其实在mTraversalRunnable回调里
/frameworks/base/core/java/android/view/ViewRootImpl.java
final class TraversalRunnable implements Runnable { @Override public void run() { doTraversal(); } } void doTraversal() { if (mTraversalScheduled) { mTraversalScheduled = false; mHandler.getLooper().getQueue().removeSyncBarrier(mTraversalBarrier); performTraversals(); if (mProfile) { Debug.stopMethodTracing(); mProfile = false; } } }mTraversalRunnable回调函数执行了performTraversals函数,这个函数主要是用来进行布局,测量和绘制的,接着看这个函数。
/frameworks/base/core/java/android/view/ViewRootImpl.java
private void performTraversals() { final View host = mView; if (host == null || !mAdded) return; // 窗口参数初始化和调整 …… boolean layoutRequested = mLayoutRequested && (!mStopped || mReportNextDraw); if (layoutRequested) { …… //1,预测量 windowSizeMayChange |= measureHierarchy(host, lp, res, desiredWindowWidth, desiredWindowHeight); } …… if (mApplyInsetsRequested) { mApplyInsetsRequested = false; if (mLayoutRequested) { // window窗口发生变化后重新测量 windowSizeMayChange |= measureHierarchy(host, lp, mView.getContext().getResources(), desiredWindowWidth, desiredWindowHeight); } } //判断窗口是否需要重新调整大小 …… final int surfaceGenerationId = mSurface.getGenerationId(); final boolean isViewVisible = viewVisibility == View.VISIBLE; final boolean windowRelayoutWasForced = mForceNextWindowRelayout; //第一次加载,或者 window 需要重新调整大小,或者 insets 发生了改变,或者 view 可见性变化了 if (mFirst || windowShouldResize || insetsChanged || viewVisibilityChanged || params != null || mForceNextWindowRelayout) { mForceNextWindowRelayout = false; …… boolean hadSurface = mSurface.isValid(); try { …… //2,调用wms,创建native Surface等工作。 relayoutResult = relayoutWindow(params, viewVisibility, insetsPending); // 判断 surface 大小等数据是否发生变化 …… //Surface的处理 …… } catch (RemoteException e) { } if (!mStopped || mReportNextDraw) { boolean focusChangedDueToTouchMode = ensureTouchModeLocally( (relayoutResult&WindowManagerGlobal.RELAYOUT_RES_IN_TOUCH_MODE) != 0); //touch mode以及之前预测量的宽高不一样,则再次进行测量 if (focusChangedDueToTouchMode || mWidth != host.getMeasuredWidth() || mHeight != host.getMeasuredHeight() || contentInsetsChanged || updatedConfiguration) { int childWidthMeasureSpec = getRootMeasureSpec(mWidth, lp.width); int childHeightMeasureSpec = getRootMeasureSpec(mHeight, lp.height); //3,进行测量 performMeasure(childWidthMeasureSpec, childHeightMeasureSpec); …… layoutRequested = true; } } } else { maybeHandleWindowMove(frame); } …… final boolean didLayout = layoutRequested && (!mStopped || mReportNextDraw); if (didLayout) { //4,执行布局工作 performLayout(lp, mWidth, mHeight); //计算透明区域 …… } //回调,请求焦点,输入法处理等工作 …… boolean cancelDraw = mAttachInfo.mTreeObserver.dispatchOnPreDraw() || !isViewVisible; if (!cancelDraw && !newSurface) { …… //4,绘制工作 performDraw(); } else { …… } mIsInTraversal = false; }performTraversals函数非常的长,做的事情也非常多,它的关键流程只有下面这件事
预测量relayoutWindow,这一步会创建Layer和Surface重新进行测量布局绘制这篇文章主要讲图像缓冲区,所以测量,布局和绘制的流程都不讲了,我们这里只关心第二步,relayoutWindow是如何创建Surface的。
/frameworks/base/core/java/android/view/ViewRootImpl.java
private int relayoutWindow(WindowManager.LayoutParams params, int viewVisibility, boolean insetsPending) throws RemoteException { float appScale = mAttachInfo.mApplicationScale; boolean restore = false; int relayoutResult = mWindowSession.relayout( mWindow, mSeq, params, (int) (mView.getMeasuredWidth() * appScale + 0.5f), (int) (mView.getMeasuredHeight() * appScale + 0.5f), viewVisibility, insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0, mWinFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets, mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame, mPendingMergedConfiguration, mSurface); return relayoutResult; }relayoutWindow方法里面mWindowSession.relayout函数
/frameworks/base/services/core/java/com/android/server/wm/Session.java
public int relayout(IWindow window, int seq, WindowManager.LayoutParams attrs, int requestedWidth, int requestedHeight, int viewFlags, int flags, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets, Rect outVisibleInsets, Rect outStableInsets, Rect outsets, Rect outBackdropFrame, MergedConfiguration mergedConfiguration, Surface outSurface) { int res = mService.relayoutWindow(this, window, seq, attrs, requestedWidth, requestedHeight, viewFlags, flags, outFrame, outOverscanInsets, outContentInsets, outVisibleInsets, outStableInsets, outsets, outBackdropFrame, mergedConfiguration, outSurface); return res; }WindowSession的relayout函数调用了wms的relayoutWindow函数
/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java
public int relayoutWindow(Session session, IWindow client, int seq, WindowManager.LayoutParams attrs, int requestedWidth, int requestedHeight, int viewVisibility, int flags, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets, Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame, MergedConfiguration mergedConfiguration, Surface outSurface) { //权限校验 …… synchronized(mWindowMap) { WindowState win = windowForClientLocked(session, client, false); …… if (viewVisibility == View.VISIBLE && (win.mAppToken == null || win.mAttrs.type == TYPE_APPLICATION_STARTING || !win.mAppToken.isClientHidden())) { // 进行布局,以分配正确大小的surface if (win.mLayoutSeq == -1) { win.setDisplayLayoutNeeded(); mWindowPlacerLocked.performSurfacePlacement(true); } result = win.relayoutVisibleWindow(mergedConfiguration, result, attrChanges, oldVisibility); try { //创建SurfaceControl result = createSurfaceControl(outSurface, result, win, winAnimator); } catch (Exception e) { // 错误处理 …… return 0; } } else { …… } …… } return result; }relayoutWindow函数主要做了两件事情
对所有Window进行测量布局和排序,前面讲到performTraversals会进行测量布局的流程,为什么这里还要进行一次呢?因为WMS中需要对所有的窗口进行一次整体的测量,比如activity的窗口会位于状态栏窗口和虚拟按键窗口之间,所以确定activity窗口的实际大小需要wms中结合其他的窗口才能进行。执行createSurfaceControl测量和排序的流程不在这儿详说了,还是只看和Surface创建相关函数createSurfaceControl
/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java
private int createSurfaceControl(Surface outSurface, int result, WindowState win, WindowStateAnimator winAnimator) { if (!win.mHasSurface) { result |= RELAYOUT_RES_SURFACE_CHANGED; } WindowSurfaceController surfaceController; try { //1,创建SurfaceControl surfaceController = winAnimator.createSurfaceLocked(win.mAttrs.type, win.mOwnerUid); } finally { Trace.traceEnd(TRACE_TAG_WINDOW_MANAGER); } if (surfaceController != null) { //2,创建Surface surfaceController.getSurface(outSurface); } else { outSurface.release(); } return result; }createSurfaceControl做了两件事情
通过createSurfaceLocked创建SurfaceController,这个过程会创建Layer通过surfaceController.getSurface创建SurfacecreateSurfaceControl实际是调用了winAnimator的createSurfaceLocked函数。winAnimator是WinState的成员变量,是主要负责窗口动画的类。我们接着看createSurfaceLocked函数
/frameworks/base/services/core/java/com/android/server/wm/WindowStateAnimator.java
WindowSurfaceController createSurfaceLocked(int windowType, int ownerUid) { final WindowState w = mWin; //…… mSurfaceController = new WindowSurfaceController(mSession.mSurfaceSession, attrs.getTitle().toString(), width, height, format, flags, this, windowType, ownerUid); //…… return mSurfaceController; }可以看到,这里以SurfaceSession为入参创建了WindowSurfaceController,SurfaceSession在前面addWindow中已经提到了,它其实就是SurfaceCompserClient。接着看WindowSurfaceController的构造函数。
/frameworks/base/services/core/java/com/android/server/wm/WindowSurfaceController.java
public WindowSurfaceController(SurfaceSession s, String name, int w, int h, int format, int flags, WindowStateAnimator animator, int windowType, int ownerUid) { mAnimator = animator; mSurfaceW = w; mSurfaceH = h; title = name; mService = animator.mService; final WindowState win = animator.mWin; mWindowType = windowType; mWindowSession = win.mSession; mSurfaceControl = new SurfaceControl( s, name, w, h, format, flags, windowType, ownerUid); if (mService.mRoot.mSurfaceTraceEnabled) { mSurfaceControl = new RemoteSurfaceTrace( mService.mRoot.mSurfaceTraceFd.getFileDescriptor(), mSurfaceControl, win); } }WindowSurfaceController的构造函数里面实际是以SurfaceSession为入参创建了SurfaceControl。
SurfaceControl的构造函数如下
/frameworks/base/core/java/android/view/SurfaceControl.java
public SurfaceControl(SurfaceSession session, String name, int w, int h, int format, int flags, SurfaceControl parent, int windowType, int ownerUid) throws OutOfResourcesException { mName = name; mNativeObject = nativeCreate(session, name, w, h, format, flags, parent != null ? parent.mNativeObject : 0, windowType, ownerUid); …… }这个SurfaceControl是java层的SurfaceControl,它在构造函数中通过jni方法nativeCreate创建了Native层的SurfaceControl。
接着看nativeCreated的实现。
/frameworks/base/core/jni/android_view_SurfaceControl.cpp
static jlong nativeCreate(JNIEnv* env, jclass clazz, jobject sessionObj, jstring nameStr, jint w, jint h, jint format, jint flags, jlong parentObject, jint windowType, jint ownerUid) { ScopedUtfChars name(env, nameStr); //通过SurfaceSession获取SurfaceComposerClient sp<SurfaceComposerClient> client(android_view_SurfaceSession_getClient(env, sessionObj)); SurfaceControl *parent = reinterpret_cast<SurfaceControl*>(parentObject); //调用createSurface,创建Layer sp<SurfaceControl> surface = client->createSurface( String8(name.c_str()), w, h, format, flags, parent, windowType, ownerUid); if (surface == NULL) { jniThrowException(env, OutOfResourcesException, NULL); return 0; } surface->incStrong((void *)nativeCreate); return reinterpret_cast<jlong>(surface.get()); }在JNI方法nativeCreate中,我们发现了createSurface函数,在这里,Layer便创建好了,剩下的只剩调用getSurface创建Surface了。
接着回到createSurfaceLocked流程中的第二步:执行surfaceController.getSurface()函数
/frameworks/base/services/core/java/com/android/server/wm/WindowSurfaceController.java
void getSurface(Surface outSurface) { outSurface.copyFrom(mSurfaceControl); }这里的outSurface是前面ViewRootImpl创建的Surface,它这个时候的还只是一个空的Surface,需要SurfaceComposerClient在native层创建真正的Surface后,将Native层的Surface和这个Java层空的Surface绑定之后,ViewRootImpl中的Surface才是一个可用的Surface。
接着看copyFrom的实现
/frameworks/base/core/java/android/view/Surface.java
public void copyFrom(SurfaceControl other) { long surfaceControlPtr = other.mNativeObject; //调用native方法创建Surface long newNativeObject = nativeGetFromSurfaceControl(surfaceControlPtr); synchronized (mLock) { if (mNativeObject != 0) { nativeRelease(mNativeObject); } //将native层的Surface和Java层的Surface绑定 setNativeObjectLocked(newNativeObject); } }接着看native函数nativeGetFromSurfaceControl
/frameworks/base/core/jni/android_view_Surface.cpp
static jlong nativeGetFromSurfaceControl(JNIEnv* env, jclass clazz, jlong surfaceControlNativeObj) { sp<SurfaceControl> ctrl(reinterpret_cast<SurfaceControl *>(surfaceControlNativeObj)); //创建Surface sp<Surface> surface(ctrl->getSurface()); if (surface != NULL) { surface->incStrong(&sRefBaseOwner); } return reinterpret_cast<jlong>(surface.get()); }在nativeGetFromSurfaceControl函数的实现中,我们终于看到缓冲区创建流程的最后一步SurfaceControl->getSurface(),至此,在Activity的绘制流程中,Layer和Surface都已经创建完成了。
这里再总结一下Activity界面显示中缓冲区的创建流程,ViewRootImpl的setView函数中会执行**requestLayout()**和 mWindowSession.addToDisplay(),其中,在mWindowSession.addToDisplay()流程中,会调用WMS来创建SurfaceComponentClient;requestLayout()流程中会使用SurfaceComponentClient来创建Layer和Surface,Layer的创建函数为SurfaceCompoentClient.createSurface(),Surface的创建函数为SurfaceCompoentClient.getSurface。创建好了图像缓冲区后,requestLayout()就可以接着进行测量布局和绘制的操作流程了。
讲完了Surface和Layer的创建,我们接着看如何创建BufferQueue。
在前面分析Layer的创建时,有提到会在Layer的初始化函数onFirstRef中会创建BufferQueue,onFirstRef函数实现如下
/frameworks/native/services/surfaceflinger/Layer.cpp
void Layer::onFirstRef() { // Creates a custom BufferQueue for SurfaceFlingerConsumer to use sp<IGraphicBufferProducer> producer; sp<IGraphicBufferConsumer> consumer; BufferQueue::createBufferQueue(&producer, &consumer, true); mProducer = new MonitoredProducer(producer, mFlinger, this); mSurfaceFlingerConsumer = new SurfaceFlingerConsumer(consumer, mTextureName, this); mSurfaceFlingerConsumer->setConsumerUsageBits(getEffectiveUsage(0)); mSurfaceFlingerConsumer->setContentsChangedListener(this); mSurfaceFlingerConsumer->setName(mName); if (mFlinger->isLayerTripleBufferingDisabled()) { mProducer->setMaxDequeuedBufferCount(2); } const sp<const DisplayDevice> hw(mFlinger->getDefaultDisplayDevice()); updateTransformHint(hw); }onFirstRef函数中关键的一步就是调用createBufferQueue创建BufferQueue,它的实现如下
/frameworks/native/libs/gui/BufferQueue.cpp
void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer, sp<IGraphicBufferConsumer>* outConsumer, bool consumerIsSurfaceFlinger) { sp<BufferQueueCore> core(new BufferQueueCore()); sp<IGraphicBufferProducer> producer(new BufferQueueProducer(core, consumerIsSurfaceFlinger)); sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core)); *outProducer = producer; *outConsumer = consumer; }createBufferQueue做了三件事情
创建BufferQueueCore创建BufferQueueProducer创建BufferQueueConsumer下面详细讲一下这三件事情
在概述里面提到过,BufferQueueCore拥有一个slots数组用来存储GraphicBuffer,并且最多可能存放64个GraphicBuffer。这里我们会进一步了解BufferQueueCore。先看看BufferQueueCore的头文件 /frameworks/native/include/gui/BufferQueueCore.h
namespace android { class BufferQueueCore : public virtual RefBase { friend class BufferQueueProducer; friend class BufferQueueConsumer; …… private: …… BufferQueueDefs::SlotsType mSlots; Fifo mQueue; std::set<int> mFreeSlots; std::list<int> mFreeBuffers; std::list<int> mUnusedSlots; std::set<int> mActiveBuffers; …… }; } #endif从BufferQueueCore的头文件可以看到,BufferQueueCore除了持有BufferQueueProducer,BufferQueueConsumer和mSlots,还持有很多其他的数据,如mQueue,mFreeSlots,mFreeBuffers等。下面介绍一下关键的几个成员变量。
mSlots就是一个BufferSlot数组,大小为NUM_BUFFER_SLOTS,也就是64个,BufferSlot主要是用来绑定GraphicBuffer的,一个BufferSlot绑定一个GraphicBuffer //文件-->/frameworks/native/include/gui/BufferQueueDefs.h namespace android { class BufferQueueCore; namespace BufferQueueDefs { typedef BufferSlot SlotsType[NUM_BUFFER_SLOTS]; } } //文件-->/frameworks/native/include/ui/BufferQueueDefs.h namespace android { namespace BufferQueueDefs { static constexpr int NUM_BUFFER_SLOTS = 64; } } mQueue:以队列的方式存放图像消费者提交的GraphicBuffermFreeSlots:代表所有没有绑定GraphicBuffer的BufferSlot集合mActiveBuffers:代表所有绑定了GraphicBuffer的BufferSlot集合,并且BufferSlot状态不为FREEmUnusedSlots:代表当前没有使用的BufferSlot集合。这里介绍一下BufferSlot,它的数据结构如下。可以看到BufferSlot持有一个GraphicBuffer,以及BufferState,还有一些其他的数据。
/frameworks/native/include/gui/BufferSlot.h
struct BufferSlot { BufferSlot() : mGraphicBuffer(nullptr), mBufferState(), ……{ } sp<GraphicBuffer> mGraphicBuffer; BufferState mBufferState; …… };BufferState又分为五种状态,FREE,DEQUEUED,QUEUED,ACQUIRED,SHARED。了解了 BufferSlot和BufferState,我们也就能理解BufferQueueCore为什么会有这么多存储GraphicBuffer的数据结构,如slots,mFreeSlots,mFreeBuffers等等,这些数据结构都是为了给BufferSlot分类,以便获取GraphicBuffer时更加高效。
了解了BufferQueueCore,我们接着看BufferQueueProducer,在前面概述中提到,Surface会中持BufferQueueProducer的BP代理。它的构造函数如下。
/frameworks/native/libs/gui/BufferQueueProducer.cpp
BufferQueueProducer::BufferQueueProducer(const sp<BufferQueueCore>& core, bool consumerIsSurfaceFlinger) : mCore(core), mSlots(core->mSlots), …… {}BufferQueueProducer的构造函数是空方法,在初始化成员变量时,会直接将前面创建好的BufferQueueCore和mSlots赋值到BufferQueueProducer的成员变量中。
我们已经知道,图像生产者通过dequeue函数来获取一块可用的GraphicBuffer,并通过queue来归还绘制了数据的Graphicbuffer。
那么我们在BufferQueueProducer中看看这两个函数的具体实现。
/frameworks/native/libs/gui/BufferQueueProducer.cpp
status_t BufferQueueProducer::dequeueBuffer(int *outSlot, sp<android::Fence> *outFence, uint32_t width, uint32_t height, PixelFormat format, uint32_t usage, FrameEventHistoryDelta* outTimestamps) { …… int found = BufferItem::INVALID_BUFFER_SLOT; while (found == BufferItem::INVALID_BUFFER_SLOT) { //1,寻找Free状态的BufferSlot status_t status = waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue,&found); } //2,把找到的slot放到mActiveBuffers中管理 if (mCore->mSharedBufferSlot != found) { mCore->mActiveBuffers.insert(found); } *outSlot = found; ATRACE_BUFFER_INDEX(found); attachedByConsumer = mSlots[found].mNeedsReallocation; mSlots[found].mNeedsReallocation = false; //修改mBufferState状态为DEQUEUE状态 mSlots[found].mBufferState.dequeue(); //3,如果找到的GraphicBuffer是空的,或者需要重新申请,则把设置到BufferSlot的参数全部初始化 if ((buffer == NULL) || buffer->needsReallocation(width, height, format, BQ_LAYER_COUNT, usage)) { mSlots[found].mAcquireCalled = false; mSlots[found].mGraphicBuffer = NULL; mSlots[found].mRequestBufferCalled = false; mSlots[found].mEglDisplay = EGL_NO_DISPLAY; mSlots[found].mEglFence = EGL_NO_SYNC_KHR; mSlots[found].mFence = Fence::NO_FENCE; mCore->mBufferAge = 0; mCore->mIsAllocating = true; returnFlags |= BUFFER_NEEDS_REALLOCATION; } else { // We add 1 because that will be the frame number when this buffer // is queued mCore->mBufferAge = mCore->mFrameCounter + 1 - mSlots[found].mFrameNumber; } …… if (returnFlags & BUFFER_NEEDS_REALLOCATION) { //4,如果GraphicBuffer为空,则重新创建GraphicBuffer,并放入对应的slot中 sp<GraphicBuffer> graphicBuffer = new GraphicBuffer( width, height, format, BQ_LAYER_COUNT, usage, {mConsumerName.string(), mConsumerName.size()}); status_t error = graphicBuffer->initCheck(); { if (error == NO_ERROR && !mCore->mIsAbandoned) { //将新创建的GraphicBuffer放入对应的BufferSLot中 graphicBuffer->setGenerationNumber(mCore->mGenerationNumber); mSlots[*outSlot].mGraphicBuffer = graphicBuffer; } …… } } …… return returnFlags; }dequeueBuffer函数做的事情主要如下
执行waitForFreeSlotThenRelock ,查找BufferQueueCore的Slot中空闲的插槽位置,并且取出BufferSlot中的GraphicBuffer的index,判断当前这个图元是否宽高,像素格式是否和请求一样,不一样则需要重新请求。
先把found的index添加到mActiveBuffer集合中,标示为活跃状态,并且设置为dequeue状态。
如果找到的BufferSlot的GraphicBuffer为空或者需要重新申请,则把设置到BufferSlot的参数全部初始化。
如果需要重新创建GraphicBuffer,则创建一个新的GraphicBuffer,接着调用GraphicBuffer的initCheck进行校验,并把新的GraphicBuffer 加入到mSlot。
了解了dequeueBuffer的流程,在接着看queueBuffer,queuebuffer是生产者使用完GraphicBuffer后把GraphicBuffer放回BufferQueue,并把BufferState修改成QUEUE状态的流程,我们看一下源码
/frameworks/native/libs/gui/BufferQueueProducer.cpp
status_t BufferQueueProducer::queueBuffer(int slot, const QueueBufferInput &input, QueueBufferOutput *output) { …… BufferItem item; { //错误处理 …… …… mSlots[slot].mFence = acquireFence; //1,修改mBufferState状态为QUEUE状态 mSlots[slot].mBufferState.queue(); // Increment the frame counter and store a local version of it // for use outside the lock on mCore->mMutex. ++mCore->mFrameCounter; currentFrameNumber = mCore->mFrameCounter; mSlots[slot].mFrameNumber = currentFrameNumber; //2,BufferItem复制 item.mAcquireCalled = mSlots[slot].mAcquireCalled; item.mGraphicBuffer = mSlots[slot].mGraphicBuffer; item.mCrop = crop; item.mTransform = transform & ~static_cast<uint32_t>(NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY); item.mTransformToDisplayInverse = (transform & NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY) != 0; item.mScalingMode = static_cast<uint32_t>(scalingMode); item.mTimestamp = requestedPresentTimestamp; item.mIsAutoTimestamp = isAutoTimestamp; item.mDataSpace = dataSpace; item.mFrameNumber = currentFrameNumber; item.mSlot = slot; item.mFence = acquireFence; item.mFenceTime = acquireFenceTime; item.mIsDroppable = mCore->mAsyncMode || mCore->mDequeueBufferCannotBlock || (mCore->mSharedBufferMode && mCore->mSharedBufferSlot == slot); item.mSurfaceDamage = surfaceDamage; item.mQueuedBuffer = true; item.mAutoRefresh = mCore->mSharedBufferMode && mCore->mAutoRefresh; mStickyTransform = stickyTransform; output->bufferReplaced = false; if (mCore->mQueue.empty()) { // BufferQueueCore的BufferSlot队列为空时,直接push到队尾 mCore->mQueue.push_back(item); frameAvailableListener = mCore->mConsumerListener; } else { // 队列不为空,需要判断一下last BufferItem是否被替换,如果可以替换就替换,如果不可以替换就直接把BufferItem放到mQueue尾部 const BufferItem& last = mCore->mQueue.itemAt( mCore->mQueue.size() - 1); if (last.mIsDroppable) { if (!last.mIsStale) { mSlots[last.mSlot].mBufferState.freeQueued(); if (!mCore->mSharedBufferMode && mSlots[last.mSlot].mBufferState.isFree()) { mSlots[last.mSlot].mBufferState.mShared = false; } if (!mSlots[last.mSlot].mBufferState.isShared()) { mCore->mActiveBuffers.erase(last.mSlot); mCore->mFreeBuffers.push_back(last.mSlot); output->bufferReplaced = true; } } // Overwrite the droppable buffer with the incoming one mCore->mQueue.editItemAt(mCore->mQueue.size() - 1) = item; frameReplacedListener = mCore->mConsumerListener; } else { mCore->mQueue.push_back(item); frameAvailableListener = mCore->mConsumerListener; } } …… } return NO_ERROR; }queueBuffer的流程主要有这两件事情:
将对应BufferSlot状态设置成QUEUED创建BufferItem对象,并将GraphicBuffer的等数据复制给BufferItem,并入队到BufferQueueCore的mQueue队列中,这样可以方便图像消费者直接按先进先出的顺序从mQueue队列取出GraphicBuffer使用了解了BufferQueueCore和BufferQueueProducer,接着看BufferQueue的最后一个元素:BufferQueueConsumer,在概览中讲到过,图像消费者SurfceFlinger通过acquire来获取图像缓冲区,通过release来释放该缓冲区。下面就看看BufferQueueConsumer中acquire和release两个操作的具体流程。
先看acquireBuffer的过程
/frameworks/native/libs/gui/BufferQueueConsumer.cpp
status_t BufferQueueConsumer::acquireBuffer(BufferItem* outBuffer, nsecs_t expectedPresent, uint64_t maxFrameNumber) { int numDroppedBuffers = 0; sp<IProducerListener> listener; { //1,检查acquire的buffer的数量是否超出了限制 …… //2.检查BufferQueueCore中的mQueue队列是否为空 if (mCore->mQueue.empty() && !sharedBufferAvailable) { return NO_BUFFER_AVAILABLE; } //获取BufferQueueCore中的mQueue队列的迭代器 BufferQueueCore::Fifo::iterator front(mCore->mQueue.begin()); int slot = BufferQueueCore::INVALID_BUFFER_SLOT; if (sharedBufferAvailable && mCore->mQueue.empty()) { //共享Buffer模式 } else { //从front获取对应的slot slot = front->mSlot; *outBuffer = *front; } if (!outBuffer->mIsStale) { mSlots[slot].mAcquireCalled = true; if (mCore->mQueue.empty()) { mSlots[slot].mBufferState.acquireNotInQueue(); } else { //将BufferState状态改为acquire mSlots[slot].mBufferState.acquire(); } mSlots[slot].mFence = Fence::NO_FENCE; } if (outBuffer->mAcquireCalled) { outBuffer->mGraphicBuffer = NULL; } //将该Buffer从mQueue中移除 mCore->mQueue.erase(front); } //回调 …… return NO_ERROR; }acquireBuffer函数中的逻辑也非常的清晰,主要就是这几件事情
判断BufferQueueCore中的mQueue是否为空,mQueue就是前面BufferQueueProducer调用queueBuffer函数时,将缓冲区入队的容器。取出对应的BufferSlot将BufferState改为acquire状态将该Buffer从mQueue中移除再接着看releaseBuffer的流程
/frameworks/native/libs/gui/BufferQueueConsumer.cpp
status_t BufferQueueConsumer::releaseBuffer(int slot, uint64_t frameNumber, const sp<Fence>& releaseFence, EGLDisplay eglDisplay, EGLSyncKHR eglFence) { //slot合法性校验 if (slot < 0 || slot >= BufferQueueDefs::NUM_BUFFER_SLOTS || releaseFence == NULL) { return BAD_VALUE; } { //slot状态校验 if (!mSlots[slot].mBufferState.isAcquired()) { return BAD_VALUE; } …… //将BufferState转为release mSlots[slot].mBufferState.release(); …… // 将slot放在FreeBuffers中 if (!mSlots[slot].mBufferState.isShared()) { mCore->mActiveBuffers.erase(slot); mCore->mFreeBuffers.push_back(slot); } } // 回调 if (listener != NULL) { listener->onBufferReleased(); } return NO_ERROR; }releaseBuffer的流程也比较简单,主要是将BufferState改为release状态,并将该BufferlSlot放在FreeBuffers这个容器中。
GraphicBuffer是图像缓冲区的最后一个组成部分,也是最基本最重要的一个单元,在前面讲到BufferQueueProucer通过dequeueBuffer获取BufferSlot时,会判断BufferSlot中的GraphicBuffer是否为空,如果为空就会创建新的GraphicBuffer。
status_t BufferQueueProducer::dequeueBuffer(int *outSlot, sp<android::Fence> *outFence, uint32_t width, uint32_t height, PixelFormat format, uint32_t usage, FrameEventHistoryDelta* outTimestamps) { …… if (returnFlags & BUFFER_NEEDS_REALLOCATION) { //如果GraphicBuffer为空,则重新创建GraphicBuffer,并放入对应的slot中 sp<GraphicBuffer> graphicBuffer = new GraphicBuffer( width, height, format, BQ_LAYER_COUNT, usage, {mConsumerName.string(), mConsumerName.size()}); status_t error = graphicBuffer->initCheck(); { if (error == NO_ERROR && !mCore->mIsAbandoned) { //将新创建的GraphicBuffer放入对应的BufferSLot中 graphicBuffer->setGenerationNumber(mCore->mGenerationNumber); mSlots[*outSlot].mGraphicBuffer = graphicBuffer; } …… } } …… return returnFlags; }可以看到在dequeueBuffer会创建GraphicBuffer,GraphicBuffer是什么呢?我们先看看它的构造函数
/frameworks/native/libs/ui/GraphicBuffer.cpp
GraphicBuffer::GraphicBuffer(uint32_t inWidth, uint32_t inHeight, PixelFormat inFormat, uint32_t inLayerCount, uint64_t usage, std::string requestorName) : GraphicBuffer() { mInitCheck = initWithSize(inWidth, inHeight, inFormat, inLayerCount, usage, std::move(requestorName)); } status_t GraphicBuffer::initWithSize(uint32_t inWidth, uint32_t inHeight, PixelFormat inFormat, uint32_t inLayerCount, uint64_t inUsage, std::string requestorName) { GraphicBufferAllocator& allocator = GraphicBufferAllocator::get(); uint32_t outStride = 0; status_t err = allocator.allocate(inWidth, inHeight, inFormat, inLayerCount, inUsage, &handle, &outStride, mId, std::move(requestorName)); if (err == NO_ERROR) { width = static_cast<int>(inWidth); height = static_cast<int>(inHeight); format = inFormat; layerCount = inLayerCount; usage = static_cast<int>(inUsage); stride = static_cast<int>(outStride); } return err; }GraphicBuffer的构造函数调用initWithSize方法创建指定大小的Buffer,流程如下
获取GraphicBufferAllocator,它是图像缓冲区的内存分配器调用GraphicBufferAllocator.allocate分配缓冲内存/frameworks/native/libs/ui/GraphicBufferAllocator.cpp
GraphicBufferAllocator::GraphicBufferAllocator() : mMapper(GraphicBufferMapper::getInstance()), mAllocator(std::make_unique<Gralloc2::Allocator>( mMapper.getGrallocMapper())) { }从GraphicBufferAllocator的构造函数中可以看到,真正的缓冲区分配器是Gralloc2::Allocator模块。由于篇幅原因,在这里就不在深入讲解Gralloc了,我们只需要了解Gralloc是位于HAL(硬件抽象层)中的模块,封装了对帧缓冲区的所有访问创建等操作。
我们已经对图像缓冲区有了一定的了解了,包括Surface,Layer,BufferQueue的创建和使用也都熟悉了。那么接下来看看图像消费者在Android中使用图像缓冲区的场景:软件绘制、硬件加速。
在图像生产者中介绍Skia时讲到过,软件绘制的入口函数drawSoftware的三步操作。
/frameworks/base/core/java/android/view/ViewRootImpl.java
private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff, boolean scalingRequired, Rect dirty) { final Canvas canvas; canvas = mSurface.lockCanvas(dirty); …… mView.draw(canvas); …… surface.unlockCanvasAndPost(canvas); …… return true; } 通过mSurface.lockCanvas获取Canvas通过draw方法,将根View及其子View遍历绘制到Canvas上通过surface.unlockCanvasAndPost将绘制内容提交给surfaceFlinger进行合成第一步会创建或获取图像缓冲区,第三步会提交图像缓冲区。
先看第一步的详细步骤
/frameworks/base/core/java/android/view/Surface.java
public Canvas lockCanvas(Rect inOutDirty) throws Surface.OutOfResourcesException, IllegalArgumentException { synchronized (mLock) { mLockedObject = nativeLockCanvas(mNativeObject, mCanvas, inOutDirty); return mCanvas; } }lockCanvas会调用native方法nativeLockCanvas
/frameworks/base/core/jni/android_view_Surface.cpp
static jlong nativeLockCanvas(JNIEnv* env, jclass clazz, jlong nativeObject, jobject canvasObj, jobject dirtyRectObj) { sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject)); if (!isSurfaceValid(surface)) { doThrowIAE(env); return 0; } Rect dirtyRect(Rect::EMPTY_RECT); Rect* dirtyRectPtr = NULL; if (dirtyRectObj) { dirtyRect.left = env->GetIntField(dirtyRectObj, gRectClassInfo.left); dirtyRect.top = env->GetIntField(dirtyRectObj, gRectClassInfo.top); dirtyRect.right = env->GetIntField(dirtyRectObj, gRectClassInfo.right); dirtyRect.bottom = env->GetIntField(dirtyRectObj, gRectClassInfo.bottom); dirtyRectPtr = &dirtyRect; } ANativeWindow_Buffer outBuffer; //1,获取用来存储图形绘制的buffer status_t err = surface->lock(&outBuffer, dirtyRectPtr); SkImageInfo info = SkImageInfo::Make(outBuffer.width, outBuffer.height, convertPixelFormat(outBuffer.format), outBuffer.format == PIXEL_FORMAT_RGBX_8888 ? kOpaque_SkAlphaType : kPremul_SkAlphaType, GraphicsJNI::defaultColorSpace()); SkBitmap bitmap; ssize_t bpr = outBuffer.stride * bytesPerPixel(outBuffer.format); bitmap.setInfo(info, bpr); if (outBuffer.width > 0 && outBuffer.height > 0) { //将上一个buffer里的图形数据复制到当前bitmap中 bitmap.setPixels(outBuffer.bits); } else { // be safe with an empty bitmap. bitmap.setPixels(NULL); } //2,创建一个SKCanvas Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj); //3,给SKCanvas设置Bitmap nativeCanvas->setBitmap(bitmap); //如果指定了脏区,则设定脏区的区域 if (dirtyRectPtr) { nativeCanvas->clipRect(dirtyRect.left, dirtyRect.top, dirtyRect.right, dirtyRect.bottom, SkClipOp::kIntersect); } if (dirtyRectObj) { env->SetIntField(dirtyRectObj, gRectClassInfo.left, dirtyRect.left); env->SetIntField(dirtyRectObj, gRectClassInfo.top, dirtyRect.top); env->SetIntField(dirtyRectObj, gRectClassInfo.right, dirtyRect.right); env->SetIntField(dirtyRectObj, gRectClassInfo.bottom, dirtyRect.bottom); } sp<Surface> lockedSurface(surface); lockedSurface->incStrong(&sRefBaseOwner); return (jlong) lockedSurface.get(); }nativeLockCanvas函数中会调用了surface->lock函数来获取一个供SKCanvas使用的缓冲区,看一下具体实现。
/frameworks/native/libs/gui/Surface.cpp
status_t Surface::lock( ANativeWindow_Buffer* outBuffer, ARect* inOutDirtyBounds) { …… ANativeWindowBuffer* out; int fenceFd = -1; //1.调用dequeueBuffer获取图像缓冲区 status_t err = dequeueBuffer(&out, &fenceFd); if (err == NO_ERROR) { sp<GraphicBuffer> backBuffer(GraphicBuffer::getSelf(out)); const Rect bounds(backBuffer->width, backBuffer->height); Region newDirtyRegion; if (inOutDirtyBounds) { newDirtyRegion.set(static_cast<Rect const&>(*inOutDirtyBounds)); newDirtyRegion.andSelf(bounds); } else { newDirtyRegion.set(bounds); } const sp<GraphicBuffer>& frontBuffer(mPostedBuffer); const bool canCopyBack = (frontBuffer != 0 && backBuffer->width == frontBuffer->width && backBuffer->height == frontBuffer->height && backBuffer->format == frontBuffer->format); if (canCopyBack) { // 2,如果前一帧的缓冲区大小和格式和当前一样,则将前一帧的缓冲区数据拷贝到当前获取的缓冲区来,这样能复用数据,节约性能。 const Region copyback(mDirtyRegion.subtract(newDirtyRegion)); if (!copyback.isEmpty()) { copyBlt(backBuffer, frontBuffer, copyback, &fenceFd); } } else { …… } …… void* vaddr; status_t res = backBuffer->lockAsync( GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN, newDirtyRegion.bounds(), &vaddr, fenceFd); …… } return err; }我们直接看dequeueBuffer是如何获取缓冲区的
/frameworks/native/libs/gui/Surface.cpp
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) { …… //创建缓冲区 status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight, reqFormat, reqUsage, enableFrameTimestamps ? &frameTimestamps : nullptr); if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == nullptr) { //映射缓冲区 result = mGraphicBufferProducer->requestBuffer(buf, &gbuf); } return OK; }dequeueBuffer关键的步骤只有两步
通过GraphicBufferProducer的dequeueBuffer获取缓冲区通过GraphicBufferProducer的requestBuffer映射缓冲区dequeueBuffer已经讲过了,就不重复讲了,我们看一下requestBuffer,当我们获取了缓冲区内存后,图像生产者和图像消费者都需要将缓冲区映射到他们的进程中才能使用。
/frameworks/native/libs/gui/IGraphicBufferProducer.cpp
virtual status_t requestBuffer(int bufferIdx, sp<GraphicBuffer>* buf) { Parcel data, reply; data.writeInterfaceToken(IGraphicBufferProducer::getInterfaceDescriptor()); data.writeInt32(bufferIdx); status_t result =remote()->transact(REQUEST_BUFFER, data, &reply); if (result != NO_ERROR) { return result; } bool nonNull = reply.readInt32(); if (nonNull) { //在用户进程创建空的GraphicBuffer *buf = new GraphicBuffer(); //通过共享内存的方式,映射GraphicBufferProducer中的GraphicBuffer result = reply.read(**buf); if(result != NO_ERROR) { (*buf).clear(); return result; } } result = reply.readInt32(); return result; }可以看到,requestBuffer函数中创建了GraphicBuffer对象,这个对应位于用户进程,并且没有申请内存。接着调用了Parcel的read的方法,将GraphicBufferProducer中的GraphicBuffer映射过来。共享内存是最高效也是传输数据量最大的IPC通信机制,关于Android IPC通信的更多知识,可以看看我的这篇文章《深入理解Android进程间通信机制》
接着看软绘是如何提交缓冲区,它的实现在nativeUnlockCanvasAndPost这个native函数中
/frameworks/base/core/jni/android_view_Surface.cpp
static void nativeUnlockCanvasAndPost(JNIEnv* env, jclass clazz, jlong nativeObject, jobject canvasObj) { sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject)); if (!isSurfaceValid(surface)) { return; } // detach the canvas from the surface Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj); nativeCanvas->setBitmap(SkBitmap()); // unlock surface status_t err = surface->unlockAndPost(); if (err < 0) { doThrowIAE(env); } }nativeUnlockCanvasAndPost调用了surface的unlockAndPost函数
/frameworks/native/libs/gui/Surface.cpp
status_t Surface::unlockAndPost() { if (mLockedBuffer == 0) { return INVALID_OPERATION; } int fd = -1; status_t err = mLockedBuffer->unlockAsync(&fd); //图像缓冲区的入队操作 err = queueBuffer(mLockedBuffer.get(), fd); mPostedBuffer = mLockedBuffer; mLockedBuffer = 0; return err; }在着看queueBuffer的实现
/frameworks/native/libs/gui/Surface.cpp
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) { Mutex::Autolock lock(mMutex); int64_t timestamp; bool isAutoTimestamp = false; //1,获取Buffer在slots中的index int i = getSlotFromBufferLocked(buffer); if (i < 0) { if (fenceFd >= 0) { close(fenceFd); } return i; } if (mSharedBufferSlot == i && mSharedBufferHasBeenQueued) { if (fenceFd >= 0) { close(fenceFd); } return OK; } …… //2,调用GraphicBufferProducer的queueBuffer函数,进行入队操作 status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output); mLastQueueDuration = systemTime() - now; if (err != OK) { ALOGE("queueBuffer: error queuing buffer to SurfaceTexture, %d", err); } …… return err; }可以看到surface.unlockCanvasAndPost最终也是通过queueBuffer函数来提交缓冲区的。
了解了软件绘制如何使用缓冲区,我们在看看硬件加速中是如何使用缓冲区的,硬件加速的绘制入口在CanvasContext的draw函数中,对硬件绘制流程不熟的,可以看图像生产者中使用OpenGL进行硬件绘制的部分。
/frameworks/base/libs/hwui/renderthread/CanvasContext.cpp
void CanvasContext::draw() { SkRect dirty; mDamageAccumulator.finish(&dirty); mCurrentFrameInfo->markIssueDrawCommandsStart(); //1,获取缓冲区 Frame frame = mRenderPipeline->getFrame(); SkRect windowDirty = computeDirtyRect(frame, &dirty); //2,调用OpenGL的draw函数进行绘制 bool drew = mRenderPipeline->draw(frame, windowDirty, dirty, mLightGeometry, &mLayerUpdateQueue, mContentDrawBounds, mOpaque, mLightInfo, mRenderNodes, &(profiler())); waitOnFences(); bool requireSwap = false; //3,提交缓冲区 bool didSwap = mRenderPipeline->swapBuffers(frame, drew, windowDirty, mCurrentFrameInfo, &requireSwap); mIsDirty = false; //…… }硬件加速和软件绘制的流程都是分为三步
获取缓冲区绘制提交缓冲区我们先看第一步:获取缓冲区。它的是在RenderPipeline的getFrame函数中,这里的RenderPipeline以OpenGLPipeline为例讲解。
/frameworks/base/libs/hwui/renderthread/OpenGLPipeline.cpp
Frame OpenGLPipeline::getFrame() { return mEglManager.beginFrame(mEglSurface); }getFrame调用了EglManager的beginFrame函数
Frame EglManager::beginFrame(EGLSurface surface) { makeCurrent(surface); Frame frame; frame.mSurface = surface; eglQuerySurface(mEglDisplay, surface, EGL_WIDTH, &frame.mWidth); eglQuerySurface(mEglDisplay, surface, EGL_HEIGHT, &frame.mHeight); frame.mBufferAge = queryBufferAge(surface); eglBeginFrame(mEglDisplay, surface); return frame; }beginFrame中的流程有这几步
调用makeCurrent获取缓冲区调用eglQuerySurface获取缓冲区的宽高调用eglBeginFrame进行参数合法性校验关键函数是makeCurrent,直接看它的实现
/frameworks/base/libs/hwui/renderthread/EglManager.cpp
bool EglManager::makeCurrent(EGLSurface surface, EGLint* errOut) { if (!eglMakeCurrent(mEglDisplay, surface, surface, mEglContext)) { if (errOut) { *errOut = eglGetError(); } else { } } mCurrentSurface = surface; if (Properties::disableVsync) { eglSwapInterval(mEglDisplay, 0); } return true; }makeCurrent实际调用的是eglMakeCurrent,这部分的实现在EGL里面
/frameworks/native/opengl/libs/EGL/egl.cpp
EGLBoolean eglMakeCurrent( EGLDisplay dpy, EGLSurface draw, EGLSurface read, EGLContext ctx) { //draw和read和合法校验 …… if (ctx == EGL_NO_CONTEXT) { current_ctx = (EGLContext)getGlThreadSpecific(); } else { egl_context_t* c = egl_context_t::context(ctx); //将EGLSurface转换成egl_surface_t egl_surface_t* d = (egl_surface_t*)draw; egl_surface_t* r = (egl_surface_t*)read; } ogles_context_t* gl = (ogles_context_t*)ctx; if (makeCurrent(gl) == 0) { if (ctx) { egl_context_t* c = egl_context_t::context(ctx); egl_surface_t* d = (egl_surface_t*)draw; egl_surface_t* r = (egl_surface_t*)read; if (c->draw) { egl_surface_t* s = reinterpret_cast<egl_surface_t*>(c->draw); s->disconnect(); s->ctx = EGL_NO_CONTEXT; if (s->zombie) delete s; } if (c->read) { } c->draw = draw; c->read = read; if (c->flags & egl_context_t::NEVER_CURRENT) { c->flags &= ~egl_context_t::NEVER_CURRENT; GLint w = 0; GLint h = 0; if (draw) { w = d->getWidth(); h = d->getHeight(); } ogles_surfaceport(gl, 0, 0); ogles_viewport(gl, 0, 0, w, h); ogles_scissor(gl, 0, 0, w, h); } if (d) { //1,调用egl_surface_t的connect函数创建缓冲区 if (d->connect() == EGL_FALSE) { return EGL_FALSE; } d->ctx = ctx; d->bindDrawSurface(gl); } if (r) { r->ctx = ctx; r->bindReadSurface(gl); } } else { //异常处理 …… } return EGL_TRUE; } return setError(EGL_BAD_ACCESS, EGL_FALSE); }eglMakeCurrent函数中我们只关心一件事情:执行d->connect()创建缓冲区
/frameworks/native/opengl/libs/EGL/egl.cpp
EGLBoolean egl_window_surface_v2_t::connect() { //获取图像缓冲区 if (nativeWindow->dequeueBuffer(nativeWindow, &buffer, &fenceFd) != NO_ERROR) { return setError(EGL_BAD_ALLOC, EGL_FALSE); } …… // 绑定缓冲区 if (lock(buffer, GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN, &bits) != NO_ERROR) { return setError(EGL_BAD_ACCESS, EGL_FALSE); } return EGL_TRUE; }可以看到connect函数中最终依然是调用了nativeWindow的dequeueBuffer函数,前面提到过Surface继承自nativeWindow,所以这里实际就是调用了Surface的dequeueBuffer函数,这个函数的实现在前面详细讲了,这儿就不再讲了。但是这里和软件绘制有点区别的地方在于缓冲区的绑定,软件绘制中是调用的requestBuffer进行共享内存的绑定,这里是调用lock函数,它的实现如下
/frameworks/native/opengl/libs/EGL/egl.cpp
status_t egl_window_surface_v2_t::lock( ANativeWindowBuffer* buf, int usage, void** vaddr) { auto& mapper = GraphicBufferMapper::get(); return mapper.lock(buf->handle, usage, android::Rect(buf->width, buf->height), vaddr); }lock函数里面其实调用GraphicBufferMapper进行了一个内存绑定的操作,GraphicBufferMapper属于Galloc模块,它是单例的,在requesetBuffer函数中进行内存共享绑定操作时,也会用到这个GraphicBufferMapper,他们是同一个GraphicBufferMapper。
我们接着看硬件加速是如何提交缓冲区的,它的实现在swapBuffers函数中。
/frameworks/native/opengl/libagl/egl.cpp
EGLBoolean egl_window_surface_v2_t::swapBuffers() { …… unlock(buffer); previousBuffer = buffer; //1,提交缓冲区 nativeWindow->queueBuffer(nativeWindow, buffer, -1); buffer = 0; int fenceFd = -1; //2,获取新的缓冲区 if (nativeWindow->dequeueBuffer(nativeWindow, &buffer, &fenceFd) == NO_ERROR) { …… } else { return setError(EGL_BAD_CURRENT_SURFACE, EGL_FALSE); } return EGL_TRUE; }swapBuffers中的逻辑很清晰,主要就两步操作:queueBuffer提交缓冲区,dequeueBuffer获取新的缓冲区。
关于缓冲区的使用实例这里就讲完了,可以看到,不管是软件绘制,还是硬件加速,使用的方式都是一致的,都为下面几步。
dequeueBuffer获取缓冲区进行缓冲区内存绑定操作使用缓冲区存储图像queueBuffer提交缓冲区这些步骤是通用的流程,不仅适用在软件绘制或者硬件绘制,还包括webview,或者是Flutter,或者我们自己开发渲染引擎,都需要使用同样的步骤。
下面接着看图像消费使用缓冲区的实例。
在图像消费者这一篇中讲到,SurfaceFlinger会在onMessageReceived收到VSync的通知
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
void SurfaceFlinger::onMessageReceived(int32_t what) { ATRACE_CALL(); switch (what) { case MessageQueue::INVALIDATE: { bool frameMissed = !mHadClientComposition && mPreviousPresentFence != Fence::NO_FENCE && (mPreviousPresentFence->getSignalTime() == Fence::SIGNAL_TIME_PENDING); ATRACE_INT("FrameMissed", static_cast<int>(frameMissed)); if (mPropagateBackpressure && frameMissed) { ALOGD("Backpressure trigger, skipping transaction & refresh!"); //如果掉帧则请求下一次VSync,跳过这一次请求 signalLayerUpdate(); break; } //更新VR模式的Flinger updateVrFlinger(); bool refreshNeeded = handleMessageTransaction(); //获取缓冲区 refreshNeeded |= handleMessageInvalidate(); refreshNeeded |= mRepaintEverything; if (refreshNeeded) { //判断是否要做刷新 signalRefresh(); } break; } case MessageQueue::REFRESH: { handleMessageRefresh(); break; } } }INVALIDATE的流程如下:
判断是否掉帧通过handleMessageTransaction函数来判断是否要处理这一次的Vsync调用handleMessageInvalidate函数获取Layer中的缓冲区执行signalRefresh函数这些流程在图像消费者都有详细讲过,我们直接看handleMessageInvalidate函数的实现
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
bool SurfaceFlinger::handleMessageInvalidate() { return handlePageFlip(); } bool SurfaceFlinger::handlePageFlip() { bool visibleRegions = false; bool frameQueued = false; bool newDataLatched = false; …… for (auto& layer : mLayersWithQueuedFrames) { //遍历所有Layer,获取缓冲区 const Region dirty(layer->latchBuffer(visibleRegions, latchTime)); layer->useSurfaceDamage(); invalidateLayerStack(layer->getLayerStack(), dirty); if (!dirty.isEmpty()) { newDataLatched = true; } } mVisibleRegionsDirty |= visibleRegions; if (frameQueued && mLayersWithQueuedFrames.empty()) { signalLayerUpdate(); } return !mLayersWithQueuedFrames.empty() && newDataLatched; }handlePageFlip函数中调用layer->latchBuffer()函数来获取缓冲区
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
Region Layer::latchBuffer(bool& recomputeVisibleRegions, nsecs_t latchTime) { status_t updateResult = mSurfaceFlingerConsumer->updateTexImage(&r, mFlinger->mPrimaryDispSync, &mAutoRefresh, &queuedBuffer, mLastFrameNumberReceived); …… return outDirtyRegion; }latchBuffer函数非常的长,流程也很多,但我们只关心它获取图像缓冲区的部分:调用SurfaceFlingerConsumer的updateTexImage函数。SurfaceFlingerConsumer是对BufferQueueConsumer的封装,我们看看updateTexImage的实现。
/frameworks/native/services/surfaceflinger/SurfaceFlingerConsumer.cpp
status_t SurfaceFlingerConsumer::updateTexImage(BufferRejecter* rejecter, const DispSync& dispSync, bool* autoRefresh, bool* queuedBuffer, uint64_t maxFrameNumber) { BufferItem item; //获取缓冲区 err = acquireBufferLocked(&item, computeExpectedPresent(dispSync), maxFrameNumber); …… // 释放前一个缓冲区 #ifdef USE_HWC2 err = updateAndReleaseLocked(item, &mPendingRelease); #else err = updateAndReleaseLocked(item); #endif if (err != NO_ERROR) { return err; } …… return err; }updateTexImage函数中主要做了两件事情:
调用acquireBufferLocked函数获取缓冲区调用releaseBufferLocked释放上一个缓冲区下面分别看一下这两件事情的实现。
acquireBufferLocked函数的实现如下:
/frameworks/native/services/surfaceflinger/SurfaceFlingerConsumer.cpp
status_t SurfaceFlingerConsumer::acquireBufferLocked(BufferItem* item, nsecs_t presentWhen, uint64_t maxFrameNumber) { status_t result = GLConsumer::acquireBufferLocked(item, presentWhen, maxFrameNumber); if (result == NO_ERROR) { mTransformToDisplayInverse = item->mTransformToDisplayInverse; mSurfaceDamage = item->mSurfaceDamage; } return result; }这里调用了GLConsumer的acquireBufferLocked,SurfaceFlingerConsumer是继承自GLConsumer,所以其实是调用父类方法。
/frameworks/native/libs/gui/ConsumerBase.cpp
status_t ConsumerBase::acquireBufferLocked(BufferItem *item, nsecs_t presentWhen, uint64_t maxFrameNumber) { if (mAbandoned) { CB_LOGE("acquireBufferLocked: ConsumerBase is abandoned!"); return NO_INIT; } //获取缓冲区 status_t err = mConsumer->acquireBuffer(item, presentWhen, maxFrameNumber); if (err != NO_ERROR) { return err; } if (item->mGraphicBuffer != NULL) { if (mSlots[item->mSlot].mGraphicBuffer != NULL) { freeBufferLocked(item->mSlot); } mSlots[item->mSlot].mGraphicBuffer = item->mGraphicBuffer; } mSlots[item->mSlot].mFrameNumber = item->mFrameNumber; mSlots[item->mSlot].mFence = item->mFence; return OK; }releaseBufferLocked函数的实现如下
/frameworks/native/libs/gui/ConsumerBase.cpp
status_t ConsumerBase::releaseBufferLocked( int slot, const sp<GraphicBuffer> graphicBuffer, EGLDisplay display, EGLSyncKHR eglFence) { if (mAbandoned) { CB_LOGE("releaseBufferLocked: ConsumerBase is abandoned!"); return NO_INIT; } //释放缓冲区 status_t err = mConsumer->releaseBuffer(slot, mSlots[slot].mFrameNumber, display, eglFence, mSlots[slot].mFence); if (err == IGraphicBufferConsumer::STALE_BUFFER_SLOT) { freeBufferLocked(slot); } mPrevFinalReleaseFence = mSlots[slot].mFence; mSlots[slot].mFence = Fence::NO_FENCE; return err; }可以看到acquireBufferLocked和releaseBufferLocked最终调用的依然是BufferQueueConsumer的acuqireBuffer和releaseBuffer函数
到这里,图像显示原理的知识就都讲完了。技术飞速发展,Android中不断有新的图像显示系统出现,如RN,小程序,以及现在很火的Flutter等等,当面对层出不穷的新技术时,我们只有掌握底层的本质原理和组成,才能快速的理解新的技术。比如我在学习Webview的时候,就将我对Android显示一个界面的基本组成(如界面的布局,测量,绘制,图像的生产者,图像的缓冲区,图像的消费)的了解迁移到对Webview的学习上,去寻找Webview在这些基本组成部分的异同点,这样很快就能熟悉和上手Webview的开发了,所以,在最后希望大家能更有耐心的深入到底层原理的学习中,而不要潜伏于技术的表面。
欢迎关注个人技术公众号,坚持更新,坚持只写高质量文章,坚持探索技术的本质。