将UIImage从BGR转换为RGB

时间:2017-08-30 19:45:00

标签: python swift opencv numpy base64

正如标题所示,我在某些UIImage色彩空间转换方面遇到了一些麻烦。 TL; DR版本是我需要一种方法将BGR格式的UIIMage转换为RGB。

这是我的应用中的事件流程:

  1. App:get Image
  2. 应用:转换为base64并发送到服务器
  3. 服务器:将base64转换为要使用的图像
  4. 服务器:将图像转换回base64字符串
  5. 服务器:将base64字符串发送到app
  6. App:将base64字符串转换为UIImage
  7. RGB version of the Test-Image on the server

    BGR version of the Test-Image client-side

    此时显示的UIImage是BGR格式。我最好的猜测是在第4步出现问题,因为在此之前图像是RGB格式的(我已将其写入文件并进行检查)。我已将代码添加到下面的第4步,仅供参考。我正在积极寻找改变UIImage客户端的色彩空间,但我并不反对修复服务器端的问题。两种解决方案都可行。

    第2步:将UIIMage转换为base64字符串

      let imageData: Data = UIImageJPEGRepresentation(map.image,0.95)!
      let base64EnCodedStr: String = imageData.base64EncodedString()
    

    步骤3:将base64字符串转换为PIL图像

    import io
    import cv2
    import base64 
    import numpy as np
    from PIL import Image
    
    # Take in base64 string and return PIL image
    def stringToImage(base64_string):
        imgdata = base64.b64decode(base64_string)
        return Image.open(io.BytesIO(imgdata))
    

    步骤4:将Image(numpy数组)转换回base64字符串

    # Convert a numpyArray into a base64 string in JPEG format
    def imageToString(npArray):
    
        # convert array to PIL Image
        newImage = Image.fromarray(npArray.astype('uint8'), 'RGB')
    
        # convert to JPEG format
        file = io.BytesIO()
        newImage.save(file, format="JPEG")
    
        # reset file pointer to start
        file.seek(0)
        img_bytes = file.read()
    
        # encode data
        encodedData = base64.b64encode(img_bytes)
    
        return encodedData.decode('ascii')
    

    编辑:

    如前所述,我可以在两个位置进行转换:Sever端或客户端。由于对此问题的回答,我能够找到两种方案的解决方案。

    解决方案1:服务器端

    参考步骤4中的代码,将该函数中的第一行更改为以下内容:

     # convert array to PIL Image
     newImage = Image.fromarray( npArray[...,[2,1,0]] ) # swap color channels which converts BGR -> RGB
    

    解决方案2:客户端

    请参阅@dfd&#39的解决方案。它编写得很好,效果非常好。这是我在我的应用程序中测试的稍微改进的版本(使用swift 4)。

    let data =  NSData(base64Encoded: base64String, options: .ignoreUnknownCharacters)
    
    let uiInput = UIImage(data: data! as Data)
    let ciInput = CIImage(image: uiInput!)
    let ctx = CIContext(options: nil)
    let swapKernel = CIColorKernel( string:
        "kernel vec4 swapRedAndGreenAmount(__sample s) {" +
                     "return s.bgra;" +
         "}"
    )
    let ciOutput = swapKernel?.apply(withExtent: (ciInput?.extent)!, arguments: [ciInput as Any])
    let cgImage = ctx.createCGImage(ciOutput!, from: (ciInput?.extent)!)
    let rgbOutput = UIImage(cgImage: cgImage!)
    

3 个答案:

答案 0 :(得分:3)

Here's a very simple CIKernel to swap things:

create table t ( ID Int , StartDate Date , EndDate Date , Flag Int ) 
insert into t ( ID , StartDate , EndDate ) Values 
 (1,'2017-01-01','2017-02-01')
,(1,'2017-01-09','2017-01-28')
,(1,'2017-04-01','2017-04-30')
,(1,'2017-04-05','2017-05-20')
,(1,'2017-04-20','2017-06-12')
,(2,'2017-06-02','2017-06-20')
,(2,'2017-06-14','2017-07-31')
,(2,'2017-06-14','2017-07-31')
,(2,'2017-06-19','2017-07-31')
,(2,'2017-06-19','2017-07-31')
,(3,'2017-01-01','2017-02-01')
,(3,'2017-02-01','2017-02-28')
,(3,'2017-04-01','2017-04-30')
,(3,'2017-06-01','2017-05-20')
,(3,'2017-08-01','2017-06-12')

Here's the Swift code to use it:

;with cte as (
  /* anchor = first start date for each id, flag = 1 */
  select t.id, t.startdate, t.enddate, flag=1
  from t
  where not exists (
    select 1
    from t i
    where i.id = t.id
      and i.startdate < t.startdate
    )  
  union all
  /* recursive, get next startdate after 30 days of previous start date
    , increment flag*/
  select s.id, s.startdate, s.enddate, s.flag
  from (
    select t.id, t.startdate, t.enddate, flag=p.flag+1
      , rn = row_number() over (partition by t.id order by t.startdate)
    from t 
      inner join cte p
        on t.id = p.id
       and t.startdate > dateadd(day,30,p.startdate)
    ) s
  where s.rn=1
)
select 
    t.id
  , t.startdate
  , t.enddate
  , x.flag
from t
  cross apply (
    /* get flag for id, startdate from cte */
    select top 1 cte.flag
    from cte
    where cte.id = t.id
      and cte.startdate <= t.startdate
    order by cte.startdate desc
    ) x

Be aware of a few things:

  • This will work on devices running iOS 9 or later.
  • Second and almost as important, this uses CoreImage and the GPU. Thus, testing this on a simulator may take seconds to render. But on a device it will take microseconds.
  • I tend to use a +----+------------+------------+------+ | id | startdate | enddate | flag | +----+------------+------------+------+ | 1 | 2017-01-01 | 2017-02-01 | 1 | | 1 | 2017-01-09 | 2017-01-28 | 1 | | 1 | 2017-04-01 | 2017-04-30 | 2 | | 1 | 2017-04-05 | 2017-05-20 | 2 | | 1 | 2017-04-20 | 2017-06-12 | 2 | | 2 | 2017-06-02 | 2017-06-20 | 1 | | 2 | 2017-06-14 | 2017-07-31 | 1 | | 2 | 2017-06-14 | 2017-07-31 | 1 | | 2 | 2017-06-19 | 2017-07-31 | 1 | | 2 | 2017-06-19 | 2017-07-31 | 1 | | 3 | 2017-01-01 | 2017-02-01 | 1 | | 3 | 2017-02-01 | 2017-02-28 | 2 | | 3 | 2017-04-01 | 2017-04-30 | 3 | | 3 | 2017-06-01 | 2017-05-20 | 4 | | 3 | 2017-08-01 | 2017-06-12 | 5 | +----+------------+------------+------+ to create a kernel vec4 swapRedAndGreenAmount(__sample s) { return s.bgra; } before ending up with a let uiInput = UIImage(named: "myImage") let ciInput = CIImage(image: uiInput!) let ctx = CIContext(options: nil) let swapKernel = CIColorKernel( source: "kernel vec4 swapRedAndGreenAmount(__sample s) {" + "return s.bgra;" + "}" ) let ciOutput = swapKernel?.apply(extent: (ciInput?.extent)!, arguments: [ciInput as Any]) let cgImage = ctx.createCGImage(ciOutput!, from: (ciInput?.extent)!) let uiOutput = UIImage(cgImage: cgImage!) . You may be able to remove this step and go straight from a CIContext to a CGImage.
  • Excuse the wrapping/unwrapping, it's converted from old code. You can probably do a better job.

Explanation:

Using CoreImage "Kernel" code, which until iOS 11 could only be a subset of GLSL code, I wrote a simple UIImage that takes a pixel's RGB value and returns the pixel color as GRB.

A CIImage is optimized to work on a single pixel at a time with no access to the pixels surrounding it. Unlike that, a UIImage is optimized to "warp" a pixel based on the pixels around it. Both of these are (more or less) optimized subclasses of a CIColorKernel, which - until iOS 11 and Metal Performance Shaders - is about the closest you get to using CIColorKernel inside of CIWarpKernel.

Final edit:

What this solution does is swap a pixel's RGB one-by-one using CoreImage. It's fast because it uses the GPU, deceptively fast (because the simulator does not give you anything close to real-time performance on a device), and simple (because it swaps things from RGB to BGR).

The actual code to do this is straightforward. Hopefully it works as a start for those who want to do much larger "under the hood" things using CoreImage.

答案 1 :(得分:0)

I don't think there's a way to do it using ref or date since iOS does not give you much leeway with regards to creating custom colorspaces. However, I found something that may help using OpenCV from this article: https://sriraghu.com/2017/06/04/computer-vision-in-ios-swiftopencv/. It requires a bit of Objective-C but with a bridging header, the code will be hidden away once it's written.

  • Add a new file -> ‘Cocoa Touch Class’, name it ‘OpenCVWrapper’ and set language to Objective-C. Click Next and select Create. When it prompted to create bridging header click on the ‘Create Bridging Header’ button. Now you can observe that there are 3 files created with names: OpenCVWrapper.h, OpenCVWrapper.m, and -Bridging-Header.h. Open ‘-Bridging-Header.h’ and add the following line: #import “OpenCVWrapper.h”
  • Go to ‘OpenCVWrapper.h’ file and add the following lines of code:
date
  • Rename OpenCVWrapper.m to “OpenCVWrapper.mm” for C++ support and add the following code:
CoreImage

The minor difference from the linked article is they are converting BGR to grayscale but we are converting BGR to RGB (good thing OpenCV has tons of conversions!).

Finally...

Now that there is a bridging header to this Objective-C class you can use CoreGraphics in Swift:

#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>

@interface OpenCVWrapper: NSObject

+ (UIImage *) rgbImageFromBGRImage: (UIImage *) image;

@end

答案 2 :(得分:0)

您可以使用底层CGImage以您希望的格式创建CIImage。

func changeToRGBA8(image: UIImage) -> UIImage? {
  guard let cgImage = image.cgImage,
    let data = cgImage.dataProvider?.data else { return nil }
    let flipped = CIImage(bitmapData: data as Data,
                          bytesPerRow: cgImage.bytesPerRow,
                          size: CGSize(width: cgImage.width, height: cgImage.height),
                          format: kCIFormatRGBA8,
                          colorSpace: cgImage.colorSpace)
    return UIImage(ciImage: flipped)
}

唯一的问题是,只有UIImage首先使用CGImage创建时才有效!您也可以将其转换为CIImage然后转换为CGImage但同样适用,只有UIImage是从CIImage创建的。

如果我有更好的答案,我可以探索并发布此限制的方法。