Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot get data #30

Open
DikshaMeghwal opened this issue Apr 3, 2020 · 1 comment
Open

cannot get data #30

DikshaMeghwal opened this issue Apr 3, 2020 · 1 comment

Comments

@DikshaMeghwal
Copy link

  • When I try to download the data using script
    python download.py -c test
    I get a .zip folder that cannot be extracted (I have curl installed)
  • I download the dataset from the http://dl.yf.io/lsun/scenes/ mentioned in the code and I can now extract this file to a folder that contains data.mdb and lock.mdb files. When I try to export it using
    python data.py export ~/Downloads/test_lmdb --out_dir .
    I get error

sequence item 0: expected str instance, int found
Any help would be appreciated.

@dejanaugrenovic
Copy link

Look at this solution: #22 - It solves the issue. Basically you need in function export_images to change two lines:
image_out_path = join(image_out_dir, key + '.webp') with
image_out_path = join(image_out_dir, key.decode() + '.webp') and
open(image_out_path, 'w') as fp: with
`open(image_out_path, 'wb') as fp:

I put the export_image function in separate file and also set absolute paths. This worked for me:
export_images.py:

from __future__ import print_function
import argparse
import cv2
import lmdb
import numpy
import os
from os.path import exists, join
def export_images(db_path, out_dir, flat=False, limit=-1):
    print('Exporting', db_path, 'to', out_dir)
    env = lmdb.open(db_path, map_size=1099511627776,
                    max_readers=100, readonly=True)
    count = 0
    with env.begin(write=False) as txn:
        cursor = txn.cursor()
        for key, val in cursor:
            if not flat:
                image_out_dir = join(out_dir, '/'.join(key[:6]))
            else:
                image_out_dir = out_dir
            if not exists(image_out_dir):
                os.makedirs(image_out_dir)
            image_out_path = join(image_out_dir, key.decode() + '.webp')
            with open(image_out_path, 'wb') as fp:
                fp.write(val)
            count += 1
            if count == limit:
                break
            if count % 1000 == 0:
                print('Finished', count, 'images')
if __name__ == '__main__':
    export_images(r'C:\Users\Dejana\Documents\lsun-master\bedroom_val_lmdb',r'C:\Users\Dejana\Documents\lsun-master\data',True)


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants