{"id":1025,"date":"2015-12-31T11:58:16","date_gmt":"2015-12-31T16:58:16","guid":{"rendered":"http:\/\/josephpcohen.com\/w\/?p=1025"},"modified":"2021-01-04T12:38:05","modified_gmt":"2021-01-04T17:38:05","slug":"blindtool","status":"publish","type":"post","link":"https:\/\/josephpcohen.com\/w\/blindtool\/","title":{"rendered":"BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning"},"content":{"rendered":"<p><iframe loading=\"lazy\" title=\"BlindTool App Demo\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/UHUC4ueEiwM?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/p>\n<p>In 2015 the ability to perform object classification locally on a cell phone allowed for a new kind of tool for the blind to be developed. I developed an Android app for the blind that tells the user what the camera is looking at and vibrates based on how confident it is. When using this app the user will wave the phone around you until you feel it vibrating more and more which means you are getting closer to an object it understands. This app has been said to &#8220;represent the New Frontier of Assistive Technology&#8221; by the Massachusetts Association for the Blind and Visually Impaired and has also been called &#8220;Shazam for the world around you&#8221; by Fast Company Magazine. The app has been downloaded by over 26k users (as of Oct 2018).<\/p>\n<p>The convolutional neural network inside the app can understand 1000 classes based on ImageNet classes. This app has been tested on the Nexus 4 and Nexus 5 phones. Please report issues so I can make a note of them here.<\/p>\n<h1>Cite<\/h1>\n<p>Joseph Paul Cohen, BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning, https:\/\/github.com\/ieee8023\/blindtool, 2015<\/p>\n<h1>Links<\/h1>\n<ul>\n<li><a href=\"https:\/\/www.youtube.com\/watch?v=UHUC4ueEiwM\">YouTube Demo\u00a0Video<\/a><\/li>\n<li><a href=\"https:\/\/play.google.com\/store\/apps\/details?id=the.blindtool\">Google Play Listing<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/ieee8023\/blindtool\">Source Code<\/a><\/li>\n<li>Google Eyes-free computing group\u00a0<a href=\"https:\/\/groups.google.com\/forum\/#!msg\/eyes-free\/u3LJR076Dyc\/HleXgKXJBwAJ\">thread 1<\/a>, <a href=\"https:\/\/groups.google.com\/forum\/#!topic\/eyes-free\/HfsXKDW6jiM\">thread 2<\/a><\/li>\n<li><a href=\"https:\/\/www.seeingwithsound.com\/android.htm\"><span style=\"font-weight: 400;\">Integration with The vOICe for Android\u00a0<\/span><\/a><\/li>\n<\/ul>\n<h1>Press<\/h1>\n<p>2017<\/p>\n<ul>\n<li><a href=\"http:\/\/pcdef.com.br\/deficiencia\/visual\/tecnologias-para-deficientes-visuais\/\"><span style=\"font-weight: 400;\">(Portuguese) People with Disabilities Portal &#8211; Technologies for the visually impaired<\/span><\/a><\/li>\n<\/ul>\n<p>2016<\/p>\n<ul>\n<li><a href=\"https:\/\/www.fastcodesign.com\/3055093\/this-amazing-app-sees-objects-for-blind-people\"><span style=\"font-weight: 400;\">Fast Company &#8211; This Amazing App Sees Objects For Blind People<\/span><\/a><\/li>\n<li><a href=\"https:\/\/mabvi.wordpress.com\/2016\/01\/13\/eyes-in-your-pocket-blindtool-app-represents-the-new-frontier-of-assistive-technology\/\"><span style=\"font-weight: 400;\">Massachusetts Association for the Blind and Visually Impaired &#8211; Eyes in Your Pocket: &#8220;BlindTool&#8221; App Represents the New Frontier of Assistive Technology<\/span><\/a><\/li>\n<li><a href=\"https:\/\/sandysview1.wordpress.com\/2016\/01\/12\/commentary-using-mobile-devices-as-virtual-eyes\/\"><span style=\"font-weight: 400;\">Sandy&#8217;s View Blog &#8211; Commentary: Using Mobile Devices As Virtual Eyes<\/span><\/a><\/li>\n<li><a href=\"http:\/\/www.umassmedia.com\/news\/an-app-for-the-blind\/article_bdd94e44-c12d-11e5-b31e-4f51343bb7a1.html\"><span style=\"font-weight: 400;\">UMass Media &#8211;\u00a0An App for The Blind<\/span><\/a><\/li>\n<li><a href=\"http:\/\/hyperallergic.com\/266586\/a-new-app-attempts-to-see-for-the-blind\/\"><span style=\"font-weight: 400;\">Hyper Allergic &#8211;\u00a0A New App Attempts to See for the Blind<\/span><\/a><\/li>\n<li><a href=\"http:\/\/www.massachusetts.edu\/news\/clips\/app-blind-describes-world\"><span style=\"font-weight: 400;\">UMass Presidents Office &#8211; An app for the blind that describes the world<\/span><\/a><\/li>\n<li><a href=\"http:\/\/www.rslnmag.fr\/post\/2016\/01\/08\/blindtool-application-accessibilite-non-voyants.aspx\"><span style=\"font-weight: 400;\">Regards sur le num\u00e9rique (RSLN) Magazine &#8211; BlindTool: an application to restore sight to the blind<\/span><\/a><\/li>\n<li><a href=\"http:\/\/www.bustle.com\/articles\/133827-blindtool-an-app-that-helps-the-blind-see-identifies-and-puts-everyday-items-into-text\"><span style=\"font-weight: 400;\">Bustle \u00a0&#8211; BlindTool, An App That Helps The Blind &#8220;See,&#8221; Identifies And Puts Everyday Items Into Text<\/span><\/a><\/li>\n<li><a href=\"http:\/\/wgnradio.com\/2016\/01\/06\/the-wintrust-business-lunch-terry-savage-scott-kleinberg-and-joseph-paul-cohen\/\"><span style=\"font-weight: 400;\">WGN Radio &#8211; Interview<\/span><\/a><\/li>\n<li><a href=\"http:\/\/mentalfloss.com\/article\/73303\/smartphone-app-aims-help-blind-see\"><span style=\"font-weight: 400;\">Mental Floss &#8211; This Smartphone App Aims to Help the Blind See<\/span><\/a><\/li>\n<li><a href=\"https:\/\/futurism.com\/could-scientists-create-dragons-using-crispr-gene-editing-bbc-news\"><span style=\"font-weight: 400;\">Futurism &#8211; Phone App Provides a &#8220;Sense of Vision&#8221; to Blind Users<\/span><\/a><\/li>\n<li><a href=\"http:\/\/laughingsquid.com\/blindtool-a-smartphone-app-that-audibly-identifies-objects-using-the-phones-camera-and-a-neural-network\/\"><span style=\"font-weight: 400;\">Laughing Squid &#8211; BlindTool, A Smartphone App That Audibly Identifies Objects Using the Phone\u2019s Camera and a Neural Network<\/span><\/a><\/li>\n<li><a href=\"http:\/\/jactiv.ouest-france.fr\/ils-sactivent\/initiatives\/blindtool-lapplication-qui-decrit-monde-pour-aveugles-58670\"><span style=\"font-weight: 400;\">(French) BlindTool: The app that describes the world for the blind<\/span><\/a><\/li>\n<li><a href=\"https:\/\/www.bostonglobe.com\/ideas\/2016\/01\/11\/app-for-blind-that-describes-world\/dEw5dfLQlSKonhNow2wbTN\/story.html\">Boston Globe &#8211; An app for the blind that describes the world<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>Another video of BlindTool in action:<\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/lpst8tVaB1E\" width=\"600\" height=\"338\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p>BlindTool watches Star Wars:<br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/52mYEkmBX30\" width=\"600\" height=\"338\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<h1>Citation<\/h1>\n<pre>@misc{Cohen2015,\r\nauthor = {Cohen, Joseph Paul},\r\ntitle = {BlindTool \u2013 A mobile app that gives a \"sense of vision\" to the blind with deep learning},\r\nurl = {https:\/\/github.com\/ieee8023\/blindtool},\r\nyear = {2015}\r\n}\r\n<\/pre>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<div class=\"mh-excerpt\"><p>In 2015 the ability to perform object classification locally on a cell phone allowed for a new kind of tool for the blind to be <a class=\"mh-excerpt-more\" href=\"https:\/\/josephpcohen.com\/w\/blindtool\/\" title=\"BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning\">[&#8230;]<\/a><\/p>\n<\/div>","protected":false},"author":1,"featured_media":1032,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[16],"tags":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>BlindTool \u2013 A mobile app that gives a &quot;sense of vision&quot; to the blind with deep learning - Joseph Paul Cohen PhD<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/josephpcohen.com\/w\/blindtool\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"BlindTool \u2013 A mobile app that gives a &quot;sense of vision&quot; to the blind with deep learning - Joseph Paul Cohen PhD\" \/>\n<meta property=\"og:description\" content=\"In 2015 the ability to perform object classification locally on a cell phone allowed for a new kind of tool for the blind to be [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/josephpcohen.com\/w\/blindtool\/\" \/>\n<meta property=\"og:site_name\" content=\"Joseph Paul Cohen PhD\" \/>\n<meta property=\"article:published_time\" content=\"2015-12-31T16:58:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-01-04T17:38:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/josephpcohen.com\/w\/wp-content\/uploads\/visually_impaired_icon_200.png\" \/>\n\t<meta property=\"og:image:width\" content=\"300\" \/>\n\t<meta property=\"og:image:height\" content=\"300\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Joseph Paul Cohen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Joseph Paul Cohen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/josephpcohen.com\/w\/blindtool\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/blindtool\/\"},\"author\":{\"name\":\"Joseph Paul Cohen\",\"@id\":\"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684\"},\"headline\":\"BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning\",\"datePublished\":\"2015-12-31T16:58:16+00:00\",\"dateModified\":\"2021-01-04T17:38:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/blindtool\/\"},\"wordCount\":446,\"publisher\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684\"},\"articleSection\":[\"Featured\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/josephpcohen.com\/w\/blindtool\/\",\"url\":\"https:\/\/josephpcohen.com\/w\/blindtool\/\",\"name\":\"BlindTool \u2013 A mobile app that gives a \\\"sense of vision\\\" to the blind with deep learning - Joseph Paul Cohen PhD\",\"isPartOf\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/#website\"},\"datePublished\":\"2015-12-31T16:58:16+00:00\",\"dateModified\":\"2021-01-04T17:38:05+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/blindtool\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/josephpcohen.com\/w\/blindtool\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/josephpcohen.com\/w\/blindtool\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/josephpcohen.com\/w\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/josephpcohen.com\/w\/#website\",\"url\":\"https:\/\/josephpcohen.com\/w\/\",\"name\":\"Joseph Paul Cohen PhD\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/josephpcohen.com\/w\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684\",\"name\":\"Joseph Paul Cohen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a810b57939e75247f570c9094e7bd16e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a810b57939e75247f570c9094e7bd16e?s=96&d=mm&r=g\",\"caption\":\"Joseph Paul Cohen\"},\"logo\":{\"@id\":\"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"BlindTool \u2013 A mobile app that gives a \"sense of vision\" to the blind with deep learning - Joseph Paul Cohen PhD","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/josephpcohen.com\/w\/blindtool\/","og_locale":"en_US","og_type":"article","og_title":"BlindTool \u2013 A mobile app that gives a \"sense of vision\" to the blind with deep learning - Joseph Paul Cohen PhD","og_description":"In 2015 the ability to perform object classification locally on a cell phone allowed for a new kind of tool for the blind to be [...]","og_url":"https:\/\/josephpcohen.com\/w\/blindtool\/","og_site_name":"Joseph Paul Cohen PhD","article_published_time":"2015-12-31T16:58:16+00:00","article_modified_time":"2021-01-04T17:38:05+00:00","og_image":[{"width":300,"height":300,"url":"https:\/\/josephpcohen.com\/w\/wp-content\/uploads\/visually_impaired_icon_200.png","type":"image\/png"}],"author":"Joseph Paul Cohen","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Joseph Paul Cohen","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/josephpcohen.com\/w\/blindtool\/#article","isPartOf":{"@id":"https:\/\/josephpcohen.com\/w\/blindtool\/"},"author":{"name":"Joseph Paul Cohen","@id":"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684"},"headline":"BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning","datePublished":"2015-12-31T16:58:16+00:00","dateModified":"2021-01-04T17:38:05+00:00","mainEntityOfPage":{"@id":"https:\/\/josephpcohen.com\/w\/blindtool\/"},"wordCount":446,"publisher":{"@id":"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684"},"articleSection":["Featured"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/josephpcohen.com\/w\/blindtool\/","url":"https:\/\/josephpcohen.com\/w\/blindtool\/","name":"BlindTool \u2013 A mobile app that gives a \"sense of vision\" to the blind with deep learning - Joseph Paul Cohen PhD","isPartOf":{"@id":"https:\/\/josephpcohen.com\/w\/#website"},"datePublished":"2015-12-31T16:58:16+00:00","dateModified":"2021-01-04T17:38:05+00:00","breadcrumb":{"@id":"https:\/\/josephpcohen.com\/w\/blindtool\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/josephpcohen.com\/w\/blindtool\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/josephpcohen.com\/w\/blindtool\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/josephpcohen.com\/w\/"},{"@type":"ListItem","position":2,"name":"BlindTool \u2013 A mobile app that gives a &#8220;sense of vision&#8221; to the blind with deep learning"}]},{"@type":"WebSite","@id":"https:\/\/josephpcohen.com\/w\/#website","url":"https:\/\/josephpcohen.com\/w\/","name":"Joseph Paul Cohen PhD","description":"","publisher":{"@id":"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/josephpcohen.com\/w\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/e25d0d5746952220f35d182ca7aa8684","name":"Joseph Paul Cohen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/a810b57939e75247f570c9094e7bd16e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a810b57939e75247f570c9094e7bd16e?s=96&d=mm&r=g","caption":"Joseph Paul Cohen"},"logo":{"@id":"https:\/\/josephpcohen.com\/w\/#\/schema\/person\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/posts\/1025"}],"collection":[{"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/comments?post=1025"}],"version-history":[{"count":25,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/posts\/1025\/revisions"}],"predecessor-version":[{"id":1562,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/posts\/1025\/revisions\/1562"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/media\/1032"}],"wp:attachment":[{"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/media?parent=1025"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/categories?post=1025"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/josephpcohen.com\/w\/wp-json\/wp\/v2\/tags?post=1025"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}