Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Overview

Deep Learning Dataset Maker

Python 3.8 QGIS 3.16.13

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

image

How to use

  1. Download and install QGIS and clone the repo :
git clone [email protected]:deepbands/deep-learning-datasets-maker.git
  1. Copy folder named deep-learning-datasets-maker in QGIS configuration folder and choose the plugin from plugin manager in QGIS (If not appeared restart QGIS).

    • You can know this folder from QGIS Setting Menu at the top-left of QGIS UI Settings > User Profiles > Open Active Profile Folder .
    • Go to python/plugins then paste the deep-learning-datasets-maker folder.
    • Full path should be like : C:\Users\$USER\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\deep-learning-datasets-maker.
  2. Open QGIS, load your raster and vector data then select the output paths for rasterized, images and labels then click ok.

TODO

  • Fix: If vector layer saved in memory not in file, rasterize can't work.
  • Fix: Splitiing Image Size.
Comments
  • Error running Quick Queries with QuickOSM recently started on Mac

    Error running Quick Queries with QuickOSM recently started on Mac

    I have been trying to run a query that I have previously had no problem running at all and have run into this error every time. Here are my system specs and whatnot:

    QGIS version | 3.22.4-Białowieża QGIS code revision 3f4577ce6e Qt version | 5.14.2 Python version | 3.8.7 GDAL/OGR version | 3.2.1 PROJ version | 6.3.2 EPSG Registry database version | v9.8.6 (2020-01-22) GEOS version | 3.9.1-CAPI-1.14.2 SQLite version | 3.31.1 PostgreSQL client version | 12.3 SpatiaLite version | 4.3.0a QWT version | 6.1.4 QScintilla2 version | 2.11.4 OS version | macOS 11.0

    Active Python plugins DigitizingTools | 1.5.1 QuickOSM | 2.0.1 quick_map_services | 0.19.29 usgs_stream_mapper | 0.2 sagaprovider | 2.12.99 grassprovider | 2.12.99 db_manager | 0.1.20 MetaSearch | 0.3.5

    And this is the dialogue from the QuickOSM Panel

    2022-03-19T18:47:06     INFO    All OSM objects with the key 'natural'='water' in the canvas or layer extent are going to be downloaded.
    2022-03-19T18:47:06     INFO    Query: natural_water
    2022-03-19T18:47:06     INFO    Encoded URL: https://lz4.overpass-api.de/api/interpreter?data=[out:xml] [timeout:25];%0A(%0A node[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A way[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A relation[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A);%0A(._;%3E;);%0Aout body;&info=QgisQuickOSMPlugin
    2022-03-19T18:47:09     INFO    Request completed
    2022-03-19T18:47:09     INFO    Checking OSM file content /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     INFO    The OSM file is: /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     CRITICAL   A critical error occurred, this is the traceback:
    2022-03-19T18:47:09     CRITICAL    
    2022-03-19T18:47:09     CRITICAL    base_processing_panel.py
    2022-03-19T18:47:09     CRITICAL    Error: Algorithm qgis:checkvalidity not found
                 
    2022-03-19T18:47:09     CRITICAL     File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/base_processing_panel.py", line 47, in run
                  self._run()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/quick_query_panel.py", line 423, in _run
                  num_layers = process_quick_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 318, in process_quick_query
                  return process_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 257, in process_query
                  return open_file(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 106, in open_file
                  layers = osm_parser.processing_parse()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/parser/osm_parser.py", line 133, in processing_parse
                  validity = processing.run(
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/tools/general.py", line 108, in run
                  return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/core/Processing.py", line 171, in runAlgorithm
                  raise QgsProcessingException(msg)
    

    Finally: The Processing panel shows this dialogue each time I try a query.

    Error: Algorithm qgis:checkvalidity not found

    I have made sure my language is american english and that the startup style of the app is Macintosh. If anyone could please please help me I would be so appreciative. This function is so important to the research I am doing for my senior thesis. Thank you so much for taking time to read this.

    opened by cecilia-hampton 1
  • Init create COCO

    Init create COCO

    @Youssef-Harby The function is implemented, please check and use it. If you can, it can be merged into develop. But at present, the cutting image is repeated and can be modified 😃

    opened by geoyee 1
  • Error running in QGIS

    Error running in QGIS

    Using on a tiff image with 4 bands RGB and alpha QGIS 3.22.4

    2022-03-10T15:24:22 WARNING Traceback (most recent call last): File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 315, in run rasterize(ras_path, vec_path, output) File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\rasterize.py", line 11, in rasterize lyr = vec_ds.GetLayer() AttributeError: 'NoneType' object has no attribute 'GetLayer'

    opened by Chris-Has 1
  • Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1. Traceback (most recent call last): File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 10, in from pycocotools import mask File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) ModuleNotFoundError: No module named 'pycocotools'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 335, in startPlugin plugins[packageName] = package.classFactory(iface) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker_init_.py", line 35, in classFactory from .split_rs_data import SplitRSData File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 46, in from .utils.COCO import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO_init.py", line 1, in from .shape_to_coco import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\shape_to_coco.py", line 16, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools_init.py", line 1, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 16, in ["python3", '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']) File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\subprocess.py", line 328, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1.

    Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] QGIS version: 3.4.13-Madeira Madeira, 567300ccf1

    Python Path: C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python/plugins C:\Users\pprt\matterport\models\research C:\Users\pprt\matterport\models\research\slim C:\Program Files\QGIS 3.4\bin\python37.zip C:\PROGRA~1\QGIS3~1.4\apps\Python37\DLLs C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib C:\Program Files\QGIS 3.4\bin C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages\tf_unet-0.1.2-py3.7.egg C:\PROGRA~1\QGIS3~1.4\apps\Python37 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32\lib C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\Pythonwin C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python

    enhancement 
    opened by aldinorizaldy 3
  • An error occurred during execution of following code

    An error occurred during execution of following code

    An error occurred during execution of following code: pyplugin_installer.instance().installPlugin('deep-learning-datasets-maker', stable=False)

    Traceback (most recent call last): File "", line 1, in File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 333, in installPlugin self.processDependencies(plugin["id"]) File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 682, in processDependencies dlg = QgsPluginDependenciesDialog(plugin_id, to_install, to_upgrade, not_found) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 92, in init _make_row(data, i, name) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 63, in _make_row widget.use_stable_version = data['use_stable_version'] KeyError: 'use_stable_version'

    Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]

    QGIS version: 3.22.3-Białowieża 'Białowieża', 1628765ec7

    Python path: ['/usr/share/qgis/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins', '/usr/share/qgis/python/plugins', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins/DeepLearningTools']

    bug 
    opened by makamkkumar 1
Releases(0.2.1)
Owner
deepbands
deepbands
Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds (Local-Lip)

Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds (Local-Lip) Introduction TL;DR: We propose an efficient and trainabl

17 Dec 01, 2022
Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis

Validated, scalable, community developed variant calling, RNA-seq and small RNA analysis. You write a high level configuration file specifying your in

Blue Collar Bioinformatics 917 Jan 03, 2023
Implementation for paper: Self-Regulation for Semantic Segmentation

Self-Regulation for Semantic Segmentation This is the PyTorch implementation for paper Self-Regulation for Semantic Segmentation, ICCV 2021. Citing SR

Dong ZHANG 30 Nov 21, 2022
Unsupervised Discovery of Object Radiance Fields

Unsupervised Discovery of Object Radiance Fields by Hong-Xing Yu, Leonidas J. Guibas and Jiajun Wu from Stanford University. arXiv link: https://arxiv

Hong-Xing Yu 148 Nov 30, 2022
Picasso: A CUDA-based Library for Deep Learning over 3D Meshes

The Picasso Library is intended for complex real-world applications with large-scale surfaces, while it also performs impressively on the small-scale applications over synthetic shape manifolds. We h

97 Dec 01, 2022
A curated list of awesome game datasets, and tools to artificial intelligence in games

🎮 Awesome Game Datasets In computer science, Artificial Intelligence (AI) is intelligence demonstrated by machines. Its definition, AI research as th

Leonardo Mauro 454 Jan 03, 2023
Code to reproduce the results for Compositional Attention

Compositional-Attention This repository contains the official implementation for the paper Compositional Attention: Disentangling Search and Retrieval

Sarthak Mittal 58 Nov 30, 2022
(ICONIP 2020) MobileHand: Real-time 3D Hand Shape and Pose Estimation from Color Image

MobileHand: Real-time 3D Hand Shape and Pose Estimation from Color Image This repo contains the source code for MobileHand, real-time estimation of 3D

90 Dec 12, 2022
Computations and statistics on manifolds with geometric structures.

Geomstats Code Continuous Integration Code coverage (numpy) Code coverage (autograd, tensorflow, pytorch) Documentation Community NEWS: Geomstats is r

875 Dec 31, 2022
Project for tracking occupancy in Tel-Aviv parking lots.

Ahuzat Dibuk - Tracking occupancy in Tel-Aviv parking lots main.py This module was set-up to be executed on Google Cloud Platform. I run it every 15 m

Geva Kipper 35 Nov 22, 2022
bio_inspired_min_nets_improve_the_performance_and_robustness_of_deep_networks

Code Submission for: Bio-inspired Min-Nets Improve the Performance and Robustness of Deep Networks Run with docker To build a docker environment, chan

0 Dec 09, 2021
Official repository for the paper F, B, Alpha Matting

FBA Matting Official repository for the paper F, B, Alpha Matting. This paper and project is under heavy revision for peer reviewed publication, and s

Marco Forte 404 Jan 05, 2023
A Python library for working with arbitrary-dimension hypercomplex numbers following the Cayley-Dickson construction of algebras.

Hypercomplex A Python library for working with quaternions, octonions, sedenions, and beyond following the Cayley-Dickson construction of hypercomplex

7 Nov 04, 2022
[ICME 2021 Oral] CORE-Text: Improving Scene Text Detection with Contrastive Relational Reasoning

CORE-Text: Improving Scene Text Detection with Contrastive Relational Reasoning This repository is the official PyTorch implementation of CORE-Text, a

Jingyang Lin 18 Aug 11, 2022
Google AI Open Images - Object Detection Track: Open Solution

Google AI Open Images - Object Detection Track: Open Solution This is an open solution to the Google AI Open Images - Object Detection Track 😃 More c

minerva.ml 46 Jun 22, 2022
Code for all the Advent of Code'21 challenges mostly written in python

Advent of Code 21 Code for all the Advent of Code'21 challenges mostly written in python. They are not necessarily the best or fastest solutions but j

4 May 26, 2022
Official Implementation of SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

Official Implementation of SimIPU SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations Since

Zhyever 37 Dec 01, 2022
Compute FID scores with PyTorch.

FID score for PyTorch This is a port of the official implementation of Fréchet Inception Distance to PyTorch. See https://github.com/bioinf-jku/TTUR f

2.1k Jan 06, 2023
CrossNorm and SelfNorm for Generalization under Distribution Shifts (ICCV 2021)

CrossNorm (CN) and SelfNorm (SN) (Accepted at ICCV 2021) This is the official PyTorch implementation of our CNSN paper, in which we propose CrossNorm

100 Dec 28, 2022
[ICML 2020] DrRepair: Learning to Repair Programs from Error Messages

DrRepair: Learning to Repair Programs from Error Messages This repo provides the source code & data of our paper: Graph-based, Self-Supervised Program

Michihiro Yasunaga 155 Jan 08, 2023