Home » Android » Video capture on AR (Vuforia) with unity in Android platform

Video capture on AR (Vuforia) with unity in Android platform

Posted by: admin May 14, 2020 Leave a comment

Questions:

I’m trying to added a video capture function to an AR app.
Basically recording what’s happening on the screen, and save it as video (allowing user to share it).
The AR APP is written with the Vuforia-Unity SDK.
We have sucessfully achieve this on iOS platform.

However, we are having great difficulties with doing the same thing on the Android platform. (we are hoping to achieve this with out rooting the device)

The following is our progress:

  1. The camera is ocuppied by the Vuforia program, I cannot get access to the video stream.

  2. I’ve tried capturing a screeshot of each frame, and then combine them to some video output; but the framerate is extreamly poor (less than 1 fps). It takes 700ms to capture a screen shot.

Am I thinking from the wrong direction? Any help will be deeply appreciated!
Many thanks!
Isaac

The following is my testing code:

public void acquireScreenshot() {
    DisplayMetrics metrics = new DisplayMetrics();
    WindowManager WM = (WindowManager) MainActivity.this.getSystemService(Context.WINDOW_SERVICE);
    Display display = WM.getDefaultDisplay();
    display.getMetrics(metrics);
    int height = metrics.heightPixels; // screen height
    int width = metrics.widthPixels; // screen width
    int pixelformat = display.getPixelFormat();
    PixelFormat localPixelFormat1 = new PixelFormat();
    PixelFormat.getPixelFormatInfo(pixelformat, localPixelFormat1);
    int deepth = localPixelFormat1.bytesPerPixel;

    byte[] arrayOfByte = new byte[height* width* deepth];
    long tmp = System.currentTimeMillis();
    try {
        for(int i = 0 ; i < 10 ; i++){
            InputStream localInputStream = readAsRoot();
            DataInputStream localDataInputStream = new DataInputStream(
                    localInputStream);
            android.util.Log.e("mytest", "-----read start-------");
            localDataInputStream.readFully(arrayOfByte);
            android.util.Log.e("mytest", "-----read end-------time = "  + (System.currentTimeMillis() -tmp ));
            localInputStream.close();

            File mid = new File("/mnt/sdcard/AAA");

            if(!mid.exists()){
                mid.mkdir();
            }

            FileOutputStream out = new FileOutputStream(new File(
                    "/mnt/sdcard/AAA/"+System.currentTimeMillis()+".png"));  
            int[] tmpColor = new int[width * height];
            int r, g, b;
            tmp = System.currentTimeMillis();
            android.util.Log.e("mytest", "-----bitmap start-------");
            for (int j = 0; j < width * height * deepth; j+=deepth) {
                b = arrayOfByte[j]&0xff;
                g = arrayOfByte[j+1]&0xff;
                r = arrayOfByte[j+2]&0xff;
                tmpColor[j/deepth] = (r << 16) | (g << 8) | b |(0xff000000);
            }
            Bitmap tmpMap = Bitmap.createBitmap(tmpColor, width, height,
                    Bitmap.Config.ARGB_8888);
            android.util.Log.e("mytest", "-----bitmap end-------time = "  + (System.currentTimeMillis() -tmp ));

            tmp = System.currentTimeMillis();
            android.util.Log.e("mytest", "-----compress start-------");
            tmpMap.compress(Bitmap.CompressFormat.PNG, 100, out);
            android.util.Log.e("mytest", "-----compress end-------time = "  + (System.currentTimeMillis() -tmp ));
            out.close();
            Thread.sleep(40);
        }


    } catch (Exception e) {
        android.util.Log.e("mytest", "Exception");
        e.printStackTrace();
    }

}
How to&Answers: