Question

我是python和机器学习的新手。我成功地从Deeplearing测试了DBN.py示例。现在我想将自己的一组图像转换为mnist.pkl.gz格式

我已经从名为JPG-PNG-to-MNIST-NN-Format的项目中尝试了一些代码在github上，但它给了我idx格式我使用一些代码将此idx格式转换为mnist.pkl，但我发现应该有一个validate_set图像，该图像未以JPG-PNG-to-MNIST-NN-Format格式显示，并且我的DBN.py代码给了我错误“输入不足” 我什至尝试过 How to put my dataset in a .pkl file in the exact format and data structure used in "mnist.pkl.gz"? 但我不知道如何准备* .csv标签。这是我的代码

from PIL import Image
from numpy import genfromtxt
import gzip, cPickle
from glob import glob
import numpy as np
import pandas as pd

def dir_to_dataset(glob_files, loc_train_labels=""):
    print("Gonna process:\n\t %s"%glob_files)
    dataset = []
    for file_count, file_name in enumerate( sorted(glob(glob_files),key=len) ):
        image = Image.open(file_name)
        img = Image.open(file_name).convert('LA') #tograyscale
        pixels = [f[0] for f in list(img.getdata())]
        dataset.append(pixels)
        if file_count % 1000 == 0:
            print("\t %s files processed"%file_count)
    # outfile = glob_files+"out"
    # np.save(outfile, dataset)
    if len(loc_train_labels) > 0:
        df = pd.read_csv(loc_train_labels)
        return np.array(dataset), np.array(df["class"])
    else:
        return np.array(dataset)


Data1, y1 = dir_to_dataset("train\\*.png","train.csv")
Data2, y2 = dir_to_dataset("valid\\*.png","valid.csv")
Data3, y3 = dir_to_dataset("test\\*.png","test.csv")

# Data and labels are read 

train_set_x = Data1[:7717]
train_set_y = y1[:7717]
val_set_x = Data2[:1653]
val_set_y = y2[:1653]
test_set_x = Data3[:1654]
test_set_y = y3[:1654]


# Divided dataset into 3 parts. I had 6281 images.

train_set = train_set_x, train_set_y
val_set = val_set_x, val_set_y
test_set = test_set_x, val_set_y

dataset = [train_set, val_set, test_set]

f = gzip.open('mnist.pkl.gz','wb')
cPickle.dump(dataset, f, protocol=2)
f.close()

但是我得到这些错误

Gonna process:
         train\*.png
Traceback (most recent call last):
  File "to-mnist.py", line 27, in <module>
    Data1, y1 = dir_to_dataset("train\\*.png","train.csv")
  File "to-mnist.py", line 22, in dir_to_dataset
    return np.array(dataset), np.array(df["class"])
  File "/home/alireza/.local/lib/python2.7/site-packages/pandas/core/frame.py", line 2927, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/alireza/.local/lib/python2.7/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'class'

我认为这与我的* .csv文件有关。 * .csv文件是普通的txt文档，其类别为0和1。像这样

Answer 1

您应该将列名称添加到数据框。像这样将“ to-mnist.py”更改为第21行。

df = pd.read_csv(loc_train_labels, names = ["class"])

Answer 2

非常感谢您回答我的问题。我在GitHub上做了一个项目，并将所有数据放入其中，以为深度学习之初的像我这样的人创建mnist.pkl.gz数据集。

您可以在这里找到它 https://github.com/tikroute/mnist.pkl.gz-dataset-creator

希望它可以帮助该领域的其他学生：）

我需要帮助以我自己的图像集创建mnist.pkl.gz

2 个答案: