SlideShare a Scribd company logo
1 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Ingesting	Drone	Data	
into	Big	Data	Platforms
Timothy	Spann	(@PaasDev)
Oracle	Code	NYC	2017
https://github.com/tspannhw/IngestingDroneData
[BRK1238-NYC]
2 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Agenda
• Drones 101
• Metadata Insights
• Ingestion Formats: MQTT, Files, REST JSON, Images, FTP
and more.
• Apache NiFi
• Big Data Storage: HDFS, HBase, Phoenix, Hive
3 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Code Walk Through
• Processing Images with TensorFlow for Image Recognition
• Writing a Java 8 Processor for Sentiment Analysis
• Writing a Java 8 Processor for analyzing HTML
• Writing a Java 8 Microservice for retrieving Phoenix/Hbase
data
• Writing a Java 8 Microservice for retrieving Hive data
• Writing a NiFi flow to wire it all together
4 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
UAV / Drones
A drone is an unmanned aircraft, better known as a unmanned aerial vehicle
(UAV) or unmanned aircraft systems (UAS).
For enterprise purposes, we are thinking of serious drones that contain high
resolution video/still cameras with onboard sensors, LIDAR and GPS.
For almost all drones, they are regulated by the Federal Aviation Administration
(FAA) and there are strict regulations you must register your drone. If you aren’t
prepare for the regulations or cost, keep your drone in your house or under half
a pound.
5 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
6 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
MetaData EXIF Geotagging
Photographs taken from UAVs, the JPEGs will have extra data stored in
EXIF.
Apache NiFi can extract this information, which is very helpful.
The most important information it contains is GPS information for latitude
and longitude.
GPS Altitude Ref: Sea level
GPS Altitude: 21 metres
GPS Longitude: -73 4' 50.41
GPS Latitude: 40 45' 16.51
7 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
hdfs dfs -get	/drone/meta/Bebop2_20160920085308-0400.json
{"date":"2016-09-20T08:53:08","Compression":"JPEG","Exif	Version":"2.10","Components	Configuration":"YCbCr","file.group":"root","Compression
Type":"Baseline","Image Description":"{	"product_id":	"090C",	"uuid":	"CE112B4276E13339AFBAE10E9ED794E3",	"run_date":	"2016-09-20T083537-
0400",	"filename":	"Bebop_2_2016-09-20T085308-0400_CE112B.jpg",	"media_date":	"2016-09-20T085308-0400"	}","Number	of	
Components":"3","Component	2":"Cb	component:	Quantization	table	1,	Sampling	factors	1	horiz/1	vert","Focal Length":"1.83	mm","Component 1":"Y	component:	
Quantization	table	0,	Sampling	factors	2	horiz/2	vert","YCbCr Positioning":"Center of	pixel	array","tiff:ResolutionUnit":"Inch","uuid":"9576a956-ec32-4314-bc8b-
bd5bb43af4f8","Date/Time	Original":"2016:09:20	08:53:08","Shutter	Speed	Value":"1/249	sec","X Resolution":"72	dots	per	
inch","tiff:Make":"PARROT","path":"/","Photometric	Interpretation":"YCbCr","Component	3":"Cr	component:	Quantization	table	1,	Sampling	factors	1	horiz/1	
vert","Unique Image	ID":"[32	bytes]","F-Number":"F2.3","modified":"2016-09-20T08:53:08","Focal	Length	35":"6mm","tiff:BitsPerSample":"8","Exposure	
Program":"Program action	(high-speed	program)","GPS	Version	ID":"2.200","GPS	Latitude	Ref":"N","meta:creation-date":"2016-09-
20T08:53:08","exif:FNumber":"2.2999999166745724","GPS	Altitude	Ref":"Sea level","Exposure Time":"8597231/2147483647	sec","GPS Longitude":"-73° 4'	
50.41"","Creation-Date":"2016-09-20T08:53:08","ISO	Speed	Ratings":"426","Make":"PARROT","Orientation":"Top,	left	side	(Horizontal	/	normal)","Metering	
Mode":"(Other)","tiff:Orientation":"1","GPS	Longitude	Ref":"W","tiff:Software":"Dragon	
3.9.0","exif:FocalLength":"1.8300001180412324","filename":"Bebop2_20160920085308-0400.jpg","XMP	Value	Count":"9","geo:long":"-
73.080669","file.owner":"root","Software":"Dragon	3.9.0","Exif	Image	Height":"1088	pixels","tiff:YResolution":"72.0","Y	Resolution":"72	dots	per	inch","GPS
Latitude":"40° 45'	16.51"","dc:description":"{	"product_id":	"090C",	"uuid":	"CE112B4276E13339AFBAE10E9ED794E3",	"run_date":	"2016-09-20T083537-
0400",	"filename":	"Bebop_2_2016-09-20T085308-0400_CE112B.jpg",	"media_date":	"2016-09-20T085308-0400"	}","geo:lat":"40.754585","FlashPix	
Version":"1.00","Data	Precision":"8	bits","White Balance":"Flash","tiff:ImageLength":"1088","description":"{	"product_id":	"090C",	"uuid":	
"CE112B4276E13339AFBAE10E9ED794E3",	"run_date":	"2016-09-20T083537-0400",	"filename":	"Bebop_2_2016-09-20T085308-0400_CE112B.jpg",	
"media_date":	"2016-09-20T085308-0400"	}","dcterms:created":"2016-09-20T08:53:08","dcterms:modified":"2016-09-20T08:53:08","Last-Modified":"2016-09-
20T08:53:08","file.permissions":"rwxrwxrwx","exif:ExposureTime":"0.00400339765660623","Last-Save-Date":"2016-09-20T08:53:08","GPS	Altitude":"21	
metres","absolute.path":"/opt/demo/dronedata/","Color	Space":"Undefined","File Size":"853159	bytes","meta:save-date":"2016-09-
20T08:53:08","file.creationTime":"2016-09-20T12:53:10+0000","Date/Time	Digitized":"2016:09:20	08:53:08","File	Name":"apache-tika-
941370357006559178.tmp","Content-Type":"image/jpeg","Aperture Value":"F2.3","X-Parsed-By":"org.apache.tika.parser.DefaultParser,	
org.apache.tika.parser.jpeg.JpegParser","File	Modified	Date":"Tue Sep	20	23:33:24	UTC	2016","tiff:XResolution":"72.0","file.lastModifiedTime":"2016-09-
20T12:53:10+0000","exif:DateTimeOriginal":"2016-09-20T08:53:08","Date/Time":"2016:09:20	08:53:08","Exif	Image	Width":"1920	pixels","Image Height":"1088	
pixels","Image Width":"1920	pixels","Unknown tag	(0xc62f)":"[19	bytes]","Resolution	Unit":"Inch","tiff:Model":"Bebop	2","exif:IsoSpeedRatings":"426","Max	
Aperture	Value":"F2.3","Exposure	Mode":"Auto exposure","Model":"Bebop 2","file.lastAccessTime":"2016-09-20T23:33:24+0000","tiff:ImageWidth":"1920","White	
Balance	Mode":"Auto white	balance"}
8 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
import paho.mqtt.client as mqtt
client = mqtt.Client()
client.username_pw_set("username","password")
client.connect(“MQTT_Broker", 14162, 60)
Sources and Formats
{ "product_id": "090C", "uuid": "CE112B4276E13339AFBAE10E9ED794E3", "run_date":
"2016-09-20T083537-0400", "filename": "Bebop_2_2016-09-20T084230-
0400_CE112B.jpg", "media_date": "2016-09-20T084230-0400" }
http://localhost:9999/dronelist
9 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
ReportingDrone	Data	Ingest
- NiFi	pulls	in	BeBop 2	Drone	images
- NiFi	routes	and	parses	metadata	
from	drone	images	including	
geodata
- NiFi	uses	TensorFlow Inception	v3	to	
recognize	objects	in	image
- NiFi	stores	images,	metadata	and	
enriched	data	in	Hadoop.
- NiFi	ingests	social	and	weather	feeds
- Java	8	Processor	Runs	Sentiment
- Spring	Boot	App	Displays	Hive	Data
- Python	Sentiment	Analysis	scripts
10 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Developed	by	the	NSA	over	
the	last	8	years.		
"NSA's	innovators	work	on	
some	of	the	most	
challenging	national	security	
problems	imaginable,"	
"Commercial	enterprises	
could	use	it	to	quickly	
control,	manage,	and	
analyze	the	flow	of	
information	from	
geographically	dispersed	
sites	– creating	
comprehensive	situational	
awareness"
-- Linda	L.	Burger,		
Director	of	the	NSA
NiFi Developed by the National Security Agency
11 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
• For	agile	and	immediate	creation,	configuration,	control	of	dataflowsVisual Command	and	Control
• Ensures	trust	of	your	dataData	Lineage	(Provenance)
• Because	not	all	data	is	of	equal importanceData	Prioritization
• Since	not	all	senders/receivers/connections	work	perfectly	all	the	timeData	Buffering/Back-Pressure
• Adapt	to	different	situations with	different	requirementsControl Latency	vs	Throughput
• Security	of	data,	and	data	accessSecure Control	Plane/Data	Plane
• ScalabilityScale	out	Clustering
• Ecosystem flexibility	and	growthExtensibility
Apache	NiFi:	Designed	for	8	challenges	of	global	enterprise	dataflow
12 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
FlowFile
• Unit	of	data	moving	through	the	system
• Content	+	Attributes	(key/value	pairs)
Processor
• Performs	the	work,	can	access	FlowFiles
Connection
• Links	between	processors
• Queues	that	can	be	dynamically	prioritized
Terminology
13 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Typical NiFi Cluster Logical View
13
14 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Connecting	Data	Between	Ecosystems	Without	Coding:	180+	Processors
All	Apache	project	logos	are	trademarks	of	the	ASF	and	the	respective	projects.
Hash
Extract
Merge
Duplicate
Scan
GeoEnrich
Replace
ConvertSplit
Translate
Route	Content
Route	Context
Route	Text
Control	Rate
Distribute	Load
Generate	Table	Fetch
Jolt	Transform	JSON
Prioritized	Delivery
Encrypt
Tail
Evaluate
Execute
All	Apache	project	logos	are	trademarks	of	the	ASF	and	the	respective	projects.
Fetch
HTTP
Syslog
Email
HTML
Image
HL7
FTP
UDP
XML
SFTP
AMQP
WebSocket
15 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Hadoop	Distributed	File	System	(HDFS)
§ Fault-Tolerance	
§ Multiple	copies	provide	performance	boost	
§ Replication	Level	is	configurable
§ Full	checksums
§ Rack	awareness	
§ Files	split	into	blocks	distributed	on	three*	servers
§ Commodity	Hardware
§ Near	limitless	horizontal	scalability
§ Looks	like	Linux	File	System	
§ WebUI
Hadoop
Scalable	Storage	and	Compute
Hive	LLAP
High	Performance	SQL	Data	Mart
16 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
What Are Apache HBase and Phoenix?
Flexible	Schema
Millisecond	Latency
SQL	and	NoSQL Interfaces
Store	and	Process	Petabytes	of	Data
Scale	out	on	Commodity	Servers
Integrated	with	YARN
100%	Open	Source
On	Top	of	HDFS
YARN	:	Data	Operating	System
HBase
RegionServer
1 ° ° ° ° ° ° ° ° ° °
° ° ° ° ° ° ° ° ° ° N
HDFS
(Permanent	Data	Storage)
HBase
RegionServer
HBase
RegionServer
Flexible Schema
Extreme Low Latency
Directly Integrated with Hadoop
SQL and NoSQL Interfaces
17 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
What	Are	Apache	Phoenix	and	HBase?
Apache HBase is distributed database modeled after Google’s BigTable and designed to
provide real-time access to data in Hadoop. Apache Phoenix provides an ANSI SQL
interface to HBase.
Features:
• Real-Time	Data	Management	for	Hadoop
• PB+	Scale
• ANSI	SQL	Interface
• Secondary	Indexes
• Cross-DC	Replication
• Fine-Grained	Security	
• Develop	in	JDBC,	ODBC,	.NET,	and	more
18 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
What	Is	Apache	Hive?
Apache	Hive is	a	SQL	data	warehouse	infrastructure	that	delivers	fast,	scalable	SQL	
processing	on	Hadoop	and	in	the	Cloud.
Features:
• Extensive	SQL:2011	Support
• ACID	Transactions
• In-Memory	Caching
• Cost-Based	Optimizer
• User-Based	Dynamic	Security
• Replication	and	Disaster	Recovery
• JDBC	and	ODBC	Support
• Compatible	with	every	major	BI	Tool
• Proven	at	300+	PB	Scale
• On	Top	of	HDFS
19 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
CODE!!!!
20 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
python	classify_image.py --image_file /opt/demo/dronedata/Bebop2_20160920083655-0400.jpg
solar	dish,	solar	collector,	solar	furnace	(score	=	0.98316)
window	screen	(score	=	0.00196)
manhole	cover	(score	=	0.00070)
radiator	(score	=	0.00041)
doormat,	welcome	mat	(score	=	0.00041)
bazel-bin/tensorflow/examples/label_image/label_image --
image=/opt/demo/dronedata/Bebop2_20160920083655-0400.jpg	
tensorflow/examples/label_image/main.cc:204]	solar	dish	(577):	0.983162I	
tensorflow/examples/label_image/main.cc:204]	window	screen	(912):	0.00196204I	
tensorflow/examples/label_image/main.cc:204]	manhole	cover	(763):	0.000704005I	
tensorflow/examples/label_image/main.cc:204]	radiator	(571):	0.000408321I	
tensorflow/examples/label_image/main.cc:204]	doormat	(972):	0.000406186
TensorFlow via Python or C++ Binary (Java Library Is New!)
21 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
/opt/demo/sentiment/run.sh
python	/opt/demo/sentiment/sentiment.py "$@”
from	nltk.sentiment.vader import	SentimentIntensityAnalyzer
import	sys
sid =	SentimentIntensityAnalyzer()
ss =	sid.polarity_scores(sys.argv[1])
print('Compound	{0}	Negative	{1}	Neutral	{2}	Positive	{3}	'.format(
ss['compound'],ss['neg'],ss['neu'],ss['pos']))
or
if	ss['compound']	==	0.00: print('Neutral')
elif ss['compound']	<	0.00: print	('Negative')
else: print('Positive')
Sentiment Analysis via Python
Pretty	easy	and	I	will	show	you	
Another	example	with	
TextBlob.
You	will	need	Python	2.7	and	
PIP	installed.
22 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
https://pip.pypa.io/en/latest/installing/
http://www.nltk.org/install.html
wget https://bootstrap.pypa.io/get-pip.py
python	get-pip.py
sudo pip install -U nltk
sudo pip install -U numpy
sudo pip	install	-U	textblob
sudo python	-m	textblob.download_corpora
Installing Sentiment Analysis Libraries for Python 2.7
Installing TensorFlow for Python 2.7
See:		https://www.tensorflow.org/install/install_mac
Installation	is	getting	easier	for	TensorFlow,	but	you	will	need	build	tools	and	Python	installed.		You	need	to	find	out	if
you	have	a	GPU	supported	by	CUDA.			If	so	your	performance	will	be	greatly	improved.
23 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
24 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
25 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
26 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Installation
Download the binary from here:
http://hortonworks.com/downloads/#dataflow
Or here:
https://nifi.apache.org/download.html
Or on Mac:
brew install nifi
https://nifi.apache.org/docs/nifi-docs/html/getting-started.html#starting-
nifi
bin/nifi.sh start
27 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
à https://hortonworks.com/hadoop-tutorial/learning-ropes-apache-nifi/
à https://github.com/jfrazee/awesome-nifi
à https://dzone.com/articles/getting-started-with-apache-nifi-and-hdf
à https://nifi.apache.org/docs.html
à https://community.hortonworks.com/articles/4356/getting-started-with-nifi-
expression-language-and.html
à https://github.com/tspannhw/rpi-sensehat-mqtt-nifi
à https://github.com/tspannhw/iot-scripts
à https://dzone.com/articles/apache-nifi-10-cheatsheet
Learning	More
28 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
à https://www.parrot.com/us/drones/parrot-bebop-2#parrot-bebop-2
à https://en.wikipedia.org/wiki/Unmanned_aerial_vehicle
à http://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/label_ima
ge/README.md
à https://www.tensorflow.org/tutorials/image_recognition
à https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/java/
org/tensorflow/examples/LabelImage.java
à https://community.hortonworks.com/content/kbentry/83100/deep-learning-iot-
workflows-with-raspberry-pi-mqtt.html
à https://sites.google.com/site/pud2gpxkmlcsv/
à http://knowbeforeyoufly.org/
à https://www.faa.gov/uas/getting_started/
Resources
29 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
à Ken	Kranz – Director	of	UAS	Big	Data at	Cognizant.	@kenkranz
à Joe	Witt	- Senior	Director	of	Engineering	at	Hortonworks.	@joewitt26
à Chris	Casano – Solution	Enngineer Manager	at	Hortonworks.
à Tom	McCuch – Sr.	Technical	Director	at	Hortonworks.	@tmccuch
à Ingesting	Drone	Data	into	Big	Data	
Platformshttps://github.com/tspannhw/IngestingDroneData
Thanks
30 ©	Hortonworks	Inc.	2011	– 2017.	All	Rights	Reserved
Contact:
Timothy	Spann			@PaaSDeV
www.meetup.com/futureofdata-princeton
https://dzone.com/users/297029/bunkertor.html
community.hortonworks.com/users/9304/tspann.html

More Related Content

What's hot (20)

PPTX
Building Data Pipelines for Solr with Apache NiFi
Bryan Bende
 
PDF
Devnexus 2018 - Let Your Data Flow with Apache NiFi
Bryan Bende
 
PDF
Apache Nifi Crash Course
DataWorks Summit
 
PDF
Apache NiFi Record Processing
Bryan Bende
 
PDF
Dataflow Management From Edge to Core with Apache NiFi
DataWorks Summit
 
PPTX
Data at Scales and the Values of Starting Small with Apache NiFi & MiNiFi
Aldrin Piri
 
PPTX
The Avant-garde of Apache NiFi
DataWorks Summit/Hadoop Summit
 
PDF
Introduction to Apache NiFi 1.11.4
Timothy Spann
 
PDF
Running Apache NiFi with Apache Spark : Integration Options
Timothy Spann
 
PDF
Taking DataFlow Management to the Edge with Apache NiFi/MiNiFi
Bryan Bende
 
PDF
Dataflow Management From Edge to Core with Apache NiFi
DataWorks Summit
 
PPTX
From Zero to Data Flow in Hours with Apache NiFi
DataWorks Summit/Hadoop Summit
 
PPTX
NJ Hadoop Meetup - Apache NiFi Deep Dive
Bryan Bende
 
PPTX
MiNiFi 0.0.1 MeetUp talk
Joe Percivall
 
PPTX
Network for the Large-scale Hadoop cluster at Yahoo! JAPAN
DataWorks Summit/Hadoop Summit
 
PDF
Enterprise IIoT Edge Processing with Apache NiFi
Timothy Spann
 
PPTX
IoT with Apache MXNet and Apache NiFi and MiniFi
DataWorks Summit
 
PDF
Introduction to data flow management using apache nifi
Anshuman Ghosh
 
PPTX
Apache NiFi 1.0 in Nutshell
DataWorks Summit/Hadoop Summit
 
PPTX
Apache NiFi in the Hadoop Ecosystem
DataWorks Summit/Hadoop Summit
 
Building Data Pipelines for Solr with Apache NiFi
Bryan Bende
 
Devnexus 2018 - Let Your Data Flow with Apache NiFi
Bryan Bende
 
Apache Nifi Crash Course
DataWorks Summit
 
Apache NiFi Record Processing
Bryan Bende
 
Dataflow Management From Edge to Core with Apache NiFi
DataWorks Summit
 
Data at Scales and the Values of Starting Small with Apache NiFi & MiNiFi
Aldrin Piri
 
The Avant-garde of Apache NiFi
DataWorks Summit/Hadoop Summit
 
Introduction to Apache NiFi 1.11.4
Timothy Spann
 
Running Apache NiFi with Apache Spark : Integration Options
Timothy Spann
 
Taking DataFlow Management to the Edge with Apache NiFi/MiNiFi
Bryan Bende
 
Dataflow Management From Edge to Core with Apache NiFi
DataWorks Summit
 
From Zero to Data Flow in Hours with Apache NiFi
DataWorks Summit/Hadoop Summit
 
NJ Hadoop Meetup - Apache NiFi Deep Dive
Bryan Bende
 
MiNiFi 0.0.1 MeetUp talk
Joe Percivall
 
Network for the Large-scale Hadoop cluster at Yahoo! JAPAN
DataWorks Summit/Hadoop Summit
 
Enterprise IIoT Edge Processing with Apache NiFi
Timothy Spann
 
IoT with Apache MXNet and Apache NiFi and MiniFi
DataWorks Summit
 
Introduction to data flow management using apache nifi
Anshuman Ghosh
 
Apache NiFi 1.0 in Nutshell
DataWorks Summit/Hadoop Summit
 
Apache NiFi in the Hadoop Ecosystem
DataWorks Summit/Hadoop Summit
 

Similar to Ingesting Drone Data into Big Data Platforms (20)

PDF
Emerging tech track kovar-david-forensics-kovar
ISSA LA
 
PDF
DSD-INT 2015 - Photogrammetric workflows and use of UA VS, Francesco nex, E-s...
Deltares
 
PPTX
Introduction to Drone Tech
Anuj Magazine
 
PPTX
AI Applications in the Drone Industry.pptx
Saurabh Srivastava
 
PPTX
UAV/Drone survey and its methodology and applications
Upendra Oli
 
PDF
Enhancing drone application development using python and dronekit
Dony Riyanto
 
PDF
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
Impyeong Lee
 
PPTX
A Drone Tale by Paolo Stagno - Sec-T 2018
Paolo Stagno
 
PPTX
A Drone Tale by Paolo Stagno - Hacktivity 2018
Paolo Stagno
 
PDF
report
Shikhar Gupta
 
PDF
DJI's Drone Solutions for Smart Cities of the Future
sitecmy
 
PPTX
Intelligent robotics and drone technology-UNIT-III.pptx
Uma Kakarlapudi
 
PDF
Local Government Showcase Drones for the planning
ssusera9462d1
 
PDF
IRJET- Surveying using Drones and Providing Results by using Photogrammme...
IRJET Journal
 
PDF
IRJET - Surveying using Drones and Providing Results by using Photogrammm...
IRJET Journal
 
PPTX
Drone-based Surveying and Mapping Automation Solutions.pptx
julia smits
 
PPTX
Making Drone data open for Scientific Research
ARDC
 
PDF
Paolo Stagno - A Drone Tale: All Your Drones Belong To Us
hacktivity
 
PPTX
I'm being followed by drones
DataWorks Summit/Hadoop Summit
 
PPTX
Impact of Drones on Civil Engineering
Colin Snow
 
Emerging tech track kovar-david-forensics-kovar
ISSA LA
 
DSD-INT 2015 - Photogrammetric workflows and use of UA VS, Francesco nex, E-s...
Deltares
 
Introduction to Drone Tech
Anuj Magazine
 
AI Applications in the Drone Industry.pptx
Saurabh Srivastava
 
UAV/Drone survey and its methodology and applications
Upendra Oli
 
Enhancing drone application development using python and dronekit
Dony Riyanto
 
라이브드론맵 (Live Drone Map) - 실시간 드론 매핑 솔루션
Impyeong Lee
 
A Drone Tale by Paolo Stagno - Sec-T 2018
Paolo Stagno
 
A Drone Tale by Paolo Stagno - Hacktivity 2018
Paolo Stagno
 
DJI's Drone Solutions for Smart Cities of the Future
sitecmy
 
Intelligent robotics and drone technology-UNIT-III.pptx
Uma Kakarlapudi
 
Local Government Showcase Drones for the planning
ssusera9462d1
 
IRJET- Surveying using Drones and Providing Results by using Photogrammme...
IRJET Journal
 
IRJET - Surveying using Drones and Providing Results by using Photogrammm...
IRJET Journal
 
Drone-based Surveying and Mapping Automation Solutions.pptx
julia smits
 
Making Drone data open for Scientific Research
ARDC
 
Paolo Stagno - A Drone Tale: All Your Drones Belong To Us
hacktivity
 
I'm being followed by drones
DataWorks Summit/Hadoop Summit
 
Impact of Drones on Civil Engineering
Colin Snow
 
Ad

More from Timothy Spann (20)

PDF
14May2025_TSPANN_FromAirQualityUnstructuredData.pdf
Timothy Spann
 
PDF
Streaming AI Pipelines with Apache NiFi and Snowflake NYC 2025
Timothy Spann
 
PDF
2025-03-03-Philly-AAAI-GoodData-Build Secure RAG Apps With Open LLM
Timothy Spann
 
PDF
Conf42_IoT_Dec2024_Building IoT Applications With Open Source
Timothy Spann
 
PDF
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
Timothy Spann
 
PDF
2024Nov20-BigDataEU-RealTimeAIWithOpenSource
Timothy Spann
 
PDF
TSPANN-2024-Nov-CloudX-Adding Generative AI to Real-Time Streaming Pipelines
Timothy Spann
 
PDF
2024-Nov-BuildStuff-Adding Generative AI to Real-Time Streaming Pipelines
Timothy Spann
 
PDF
14 November 2024 - Conf 42 - Prompt Engineering - Codeless Generative AI Pipe...
Timothy Spann
 
PDF
2024 Nov 05 - Linux Foundation TAC TALK With Milvus
Timothy Spann
 
PPTX
tspann06-NOV-2024_AI-Alliance_NYC_ intro to Data Prep Kit and Open Source RAG
Timothy Spann
 
PDF
tspann08-Nov-2024_PyDataNYC_Unstructured Data Processing with a Raspberry Pi ...
Timothy Spann
 
PDF
2024-10-28 All Things Open - Advanced Retrieval Augmented Generation (RAG) Te...
Timothy Spann
 
PDF
10-25-2024_BITS_NYC_Unstructured Data and LLM_ What, Why and How
Timothy Spann
 
PDF
2024-OCT-23 NYC Meetup - Unstructured Data Meetup - Unstructured Halloween
Timothy Spann
 
PDF
DBTA Round Table with Zilliz and Airbyte - Unstructured Data Engineering
Timothy Spann
 
PDF
17-October-2024 NYC AI Camp - Step-by-Step RAG 101
Timothy Spann
 
PDF
11-OCT-2024_AI_101_CryptoOracle_UnstructuredData
Timothy Spann
 
PDF
2024-10-04 - Grace Hopper Celebration Open Source Day - Stefan
Timothy Spann
 
PDF
01-Oct-2024_PES-VectorDatabasesAndAI.pdf
Timothy Spann
 
14May2025_TSPANN_FromAirQualityUnstructuredData.pdf
Timothy Spann
 
Streaming AI Pipelines with Apache NiFi and Snowflake NYC 2025
Timothy Spann
 
2025-03-03-Philly-AAAI-GoodData-Build Secure RAG Apps With Open LLM
Timothy Spann
 
Conf42_IoT_Dec2024_Building IoT Applications With Open Source
Timothy Spann
 
2024 Dec 05 - PyData Global - Tutorial Its In The Air Tonight
Timothy Spann
 
2024Nov20-BigDataEU-RealTimeAIWithOpenSource
Timothy Spann
 
TSPANN-2024-Nov-CloudX-Adding Generative AI to Real-Time Streaming Pipelines
Timothy Spann
 
2024-Nov-BuildStuff-Adding Generative AI to Real-Time Streaming Pipelines
Timothy Spann
 
14 November 2024 - Conf 42 - Prompt Engineering - Codeless Generative AI Pipe...
Timothy Spann
 
2024 Nov 05 - Linux Foundation TAC TALK With Milvus
Timothy Spann
 
tspann06-NOV-2024_AI-Alliance_NYC_ intro to Data Prep Kit and Open Source RAG
Timothy Spann
 
tspann08-Nov-2024_PyDataNYC_Unstructured Data Processing with a Raspberry Pi ...
Timothy Spann
 
2024-10-28 All Things Open - Advanced Retrieval Augmented Generation (RAG) Te...
Timothy Spann
 
10-25-2024_BITS_NYC_Unstructured Data and LLM_ What, Why and How
Timothy Spann
 
2024-OCT-23 NYC Meetup - Unstructured Data Meetup - Unstructured Halloween
Timothy Spann
 
DBTA Round Table with Zilliz and Airbyte - Unstructured Data Engineering
Timothy Spann
 
17-October-2024 NYC AI Camp - Step-by-Step RAG 101
Timothy Spann
 
11-OCT-2024_AI_101_CryptoOracle_UnstructuredData
Timothy Spann
 
2024-10-04 - Grace Hopper Celebration Open Source Day - Stefan
Timothy Spann
 
01-Oct-2024_PES-VectorDatabasesAndAI.pdf
Timothy Spann
 
Ad

Recently uploaded (20)

PPTX
apidays Munich 2025 - Building an AWS Serverless Application with Terraform, ...
apidays
 
PPTX
apidays Singapore 2025 - Designing for Change, Julie Schiller (Google)
apidays
 
PDF
What does good look like - CRAP Brighton 8 July 2025
Jan Kierzyk
 
PPTX
apidays Helsinki & North 2025 - Running a Successful API Program: Best Practi...
apidays
 
PPTX
Module-5-Measures-of-Central-Tendency-Grouped-Data-1.pptx
lacsonjhoma0407
 
PDF
Data Chunking Strategies for RAG in 2025.pdf
Tamanna
 
PPTX
Advanced_NLP_with_Transformers_PPT_final 50.pptx
Shiwani Gupta
 
PDF
Choosing the Right Database for Indexing.pdf
Tamanna
 
PDF
Web Scraping with Google Gemini 2.0 .pdf
Tamanna
 
PDF
apidays Helsinki & North 2025 - How (not) to run a Graphql Stewardship Group,...
apidays
 
PDF
Avatar for apidays apidays PRO June 07, 2025 0 5 apidays Helsinki & North 2...
apidays
 
PPTX
ER_Model_Relationship_in_DBMS_Presentation.pptx
dharaadhvaryu1992
 
PPTX
b6057ea5-8e8c-4415-90c0-ed8e9666ffcd.pptx
Anees487379
 
PDF
Context Engineering for AI Agents, approaches, memories.pdf
Tamanna
 
PPTX
apidays Munich 2025 - Building Telco-Aware Apps with Open Gateway APIs, Subhr...
apidays
 
PDF
Merits and Demerits of DBMS over File System & 3-Tier Architecture in DBMS
MD RIZWAN MOLLA
 
PDF
How to Connect Your On-Premises Site to AWS Using Site-to-Site VPN.pdf
Tamanna
 
PDF
apidays Helsinki & North 2025 - APIs in the healthcare sector: hospitals inte...
apidays
 
PDF
Building Production-Ready AI Agents with LangGraph.pdf
Tamanna
 
PPTX
apidays Helsinki & North 2025 - API access control strategies beyond JWT bear...
apidays
 
apidays Munich 2025 - Building an AWS Serverless Application with Terraform, ...
apidays
 
apidays Singapore 2025 - Designing for Change, Julie Schiller (Google)
apidays
 
What does good look like - CRAP Brighton 8 July 2025
Jan Kierzyk
 
apidays Helsinki & North 2025 - Running a Successful API Program: Best Practi...
apidays
 
Module-5-Measures-of-Central-Tendency-Grouped-Data-1.pptx
lacsonjhoma0407
 
Data Chunking Strategies for RAG in 2025.pdf
Tamanna
 
Advanced_NLP_with_Transformers_PPT_final 50.pptx
Shiwani Gupta
 
Choosing the Right Database for Indexing.pdf
Tamanna
 
Web Scraping with Google Gemini 2.0 .pdf
Tamanna
 
apidays Helsinki & North 2025 - How (not) to run a Graphql Stewardship Group,...
apidays
 
Avatar for apidays apidays PRO June 07, 2025 0 5 apidays Helsinki & North 2...
apidays
 
ER_Model_Relationship_in_DBMS_Presentation.pptx
dharaadhvaryu1992
 
b6057ea5-8e8c-4415-90c0-ed8e9666ffcd.pptx
Anees487379
 
Context Engineering for AI Agents, approaches, memories.pdf
Tamanna
 
apidays Munich 2025 - Building Telco-Aware Apps with Open Gateway APIs, Subhr...
apidays
 
Merits and Demerits of DBMS over File System & 3-Tier Architecture in DBMS
MD RIZWAN MOLLA
 
How to Connect Your On-Premises Site to AWS Using Site-to-Site VPN.pdf
Tamanna
 
apidays Helsinki & North 2025 - APIs in the healthcare sector: hospitals inte...
apidays
 
Building Production-Ready AI Agents with LangGraph.pdf
Tamanna
 
apidays Helsinki & North 2025 - API access control strategies beyond JWT bear...
apidays
 

Ingesting Drone Data into Big Data Platforms