this is Norman Coombs from EASI and we are doing something different this week. We are going to be doing desktop sharing, John Gardner's been doing a lot of groundbreaking work on making the various aspects of math accessible and he's going to be doing that today and to do that he wants to share his desktop so if you want to see the desktop, you better say yes. John we are having this captioned if you run into technical terms, you might spell them for the captioner, and if the screenreader is talking and you cannot make sense of it, you can type into the transcript, the screenreader was talking and let it go for now. So John Gardiner I have known for 20 years away back to (inaudible) at Princeton and R&D working on science and math and it's been a long road since then but John has been one of the leaders breaking down the barriers and moving us more into a situation where people with disabilities can succeed in science and math as John does himself. He is a blind physicist. I don't know if I want to make other announcements than that. So I think I'm going to let go of the mike and let John take over.
JOHN GARDNER: Well, thank you, Norman, nice introduction. What I'm going to talk about today is a technology for making graphical information accessible. And in particular we're going to focus today on mathematical and scientific type diagrams. As I said, I'm a physicist. I founded the company's view plus technologies many people in the audience will maybe recognize it as the company that makes tiger embossers. And the tiger embosser is an essential part of what I will be showing you today but it is the only part. For those of you who can see and have at your computer working properly, what is showing on my screen right now is a scientific graph, and let me just take it a little bit through the steps that have come about to make this the first file I did not make this graph. This graph was sent to me by a friend, Kyle Keene who is also a blind physicist who works for the company Wolfram. Wolfram is the company that makes and sells the Cadillac program of mathematical computation called Mathematica. So, what Kyle did was perfectly straightforward production of a graphic, which he saved in the form of scalable vector graphics, which we will called SVG. It is a standard format. And he sent me the file. So, what I did was to open the file in this software application that I will be talking about today. This is called Iveo, for the caption is that is spelled I-V-E-O. It is a software technology that is intended to make graphical information accessible. When I started developing accessible technology tab 20 years ago my dream was to find a way to make graphics accessible directly so that a blind person could take an electronic graphical file and somehow just read it. We are still a long way from being able to do that in general but this is something that in fact could work for the type of graphs that are made by Mathematica. So if somebody took this graphic and put it on the web, a blind person would have access to it the same way that I can't access to. So I opened it in the Iveo program and went to the print bar and printed it. And what I printed it on is an old Tiger Pro embosser sitting by my decks which is what I've been using for for 10 years or so we have a much more modern version but this serves my purposes just fine so I continue to use it. I created a copy of it. He had 8.5 x 11 paper in it so that's what I chose to print on. I could have made something that was in 11 x 14 copy in for some graphics that are confused and cluttered that's a good idea. This is pretty simple and 8 1/2 by 11 is just fine. So I made the graphic and put it in a touch pad and and for those who can see in the audience what I want to do a show you how I access it. First of all I can feel it, I can feel the graphic. And I can also read any text. There is text on here. There are labels on the x-axis and y-axis and now I want to touch a few so you can listen. Let me touch some of the labels on the x-axis.
(screenreader speaking): “0.4 graphic, zero point, graphic, 0.6, graphic, 0.80 point graphic, graphic graphic”
So what you are hearing is the screenreader speaking the text, is also speaking some annoying other information, and that is partly my fault because we have recently made what is in some sense an improvement in the speaking technology because it is speaking over my screen reader, which in this case is a Window-Eyes, which is very nice and very convenient, but also comes with the difficulty that I cannot tell Window-Eyes not to speak regular events. This is a problem we are working on and hopefully we will be able to solve some code so that it will not speak regular events, it will just speak things that are directly coming from the speaker. By the way, what I am using here I believe is a beta copy of yet another Iveo creator, I usually have something beyond what is at least perversion, but is essentially identical to what is being released. So there is some text on this and in this particular one I asked Kyle to put some vertical text on because vertical text should read and I wanted to make sure it would.
“Window-Eyes speaking: graphic, squared X, graphic, squared”
JOHN GARDNER: It is saying this is X squared. With a little bit of improvement in the Mathematica software Kyle assures me that he could send that as one text string, so instead of saying this when I press this and is when I press is and Xs etc. it was because a full string this is X squared, some this would be a little bit more accessible. So basically everything on this graph speaks, and apart from the little bit annoying extra speaking that hopefully we will be able to improve soon, it provides information. Now, let me make sure you understand. No sighted person did anything to this graph. He blind physicist working from a graph using the software sent me the SVG copy electronically. I opened it, printed it, put it in there and I can read it and that's pretty nifty. I think I will pause for a minute and give it back to Norm to see if no one has any comments or instructions or anything so, Norm, you're up.
NORM COOMBS: Thank you again, John again your voice is nice and clear and the audio from your computer is coming through as well as synthetic speech can read it, so no, I have no comments. I am listening here, eager to learn more and so between some of the new things with vector graphics and the Tiger embosser, looks like we will be moving ahead, thank you, keep going.
JOHN GARDNER: Okay, I just changed the graph to another one that Kyle sent me with an equation on it, this is a fairly simple equation. It says X squared plus BX plus C13 or something like that. I can't read this equation in a sense
Window-Eyes speaking: “Graphic, X, two, graphic plus graphic, graphic graphic graphic, X plus graphic, graphic, B, graphic, graphic”
JOHN GARDNER: Okay, so I'm getting a little help from John on this one is that it pronounces each symbol, a, X, squared is a raised two just pronounced to because it's fairly large particular one I can feel that it's raised so I know that it is a squared and it read everything except for the = and = is the one that Kyle sneakily put in because he knew it would not read because it is not Unicode it's a special = that has some special meaning. I believe it is a graphic. It could be made accessible quite easily by simply giving it a title and one might not even know that he's reading. There is another one we will be doing hopefully within a reasonably short period of time. Within the next year. We want to add them math ML equivalent of this equation in some sort of special equation so when I touch this equation I don't hear the individual, I will hear the equation spoken in some fairly simple matter, so it would say a X squared plus BX plus C13 in this case. Doing something is something we know how to do it's just a matter of work and learning how to do it but it would improve the accessibility. This is accessible but it would be a lot more usable if instead of reading one letter at a time, which is doable, but especially if the characters are smaller, it gets really difficult to do. Your fingers are a lot bigger. Okay, so there is one other thing, question Kyle asks or I ask, and that is what about special symbols, Greek characters, things like that. So he made another graphic and sent to me that has special symbols on it, and I'd like to look at that and then we will go on to another topic. Okay, I'm back. Talking again. What we have now showing on the screen is the graph that Kyle sent to me that has some Unicode symbols on it. And we just wanted to see if it would work, there is a title at the top that says symbol
JOHN GARDNER: Okay I'm going to let John Taylor help me on this one, so John click on the (INAUDIBLE), Greek small letter L5, Greek small letter beta let me tell you what it is doing this is the advantage of using my screen reader because my screen printer has a built in dictionary. This thing will also where Justin self voicing if I turn the screen reader off it would automatically diverge and begin using whatever that default speech engine for Windows-Eyes is, but the speech engine presently does not have a dictionary so we need to actually put a dictionary and it would be able to speak all the Unicode symbols. But using a screenreader especially a screenreader like mine that I have put a dictionary in I will be able to make this available to other people so that it speaks basically all Unicode characters. So essentially what I'm seeing is right now almost anything that you can make with Mathematica is going to have accessible text on it and that is pretty slick, but the world is not so simple. Not everything is made with Mathematica, the world is full of lots of stuff that you have to get from paper that are available in PDF, are available in various sorts of graphic formats. So what do we do about these?
Some of these things are anything but a blind person can actually do himself and I have done this on a number of occasions, I've given thoughts on how I can open a well-made PDF and extract the figures from it. And import those figures. And a lot of the text on those figures is accessible. Usually not every single bit of it, like the cool friend graphics, but a lot of it, but on the other hand we've designed Iveo really very friendly for sighted people to use so for a sighted person to make something accessible for a blind person is a matter of relatively little work for most graphics it's an extraordinarily complex thing, might take an hour or so, but relatively simple graphics like what we are seeing now, reasonably straightforward. So I'm going to turn the mic over to John Taylor, my sighted helper who is going to bring up something and show you how we can convert an existing PDF into something that is read by Iveo.
JOHN GARDNER: Okay, thank you, real quick---
[Window-Eyes speaking]
JOHN GARDNER: How you can import PDF
John Taylor speaking: Okay, sorry I was trying to find it really fast, it takes just a second here and it will import. And you can see that I imported a small grass, and this was scanned out of a textbook. Some of this text is already accessible from the importer and other text I can make accessible by simply adding text to the document or creating overlays, so how you do that is you click add, text (inaudible) I am going to call this, okay, so what I did was simply overrode some of the text with text from Iveo so when you touch it now when it is embossed, it will now be read to you and it is really as simple as that, you can also create what is called an overlay, something that does not have text, you can actually assign some text to the graphic itself.
[Window-Eyes reading]
John Taylor: So what I did was insert what was called an overlay on the top of the graphs I'm going to give that a name, so I will call it simply graph
Okay now and I put an overlay over the top of the graph and if I click on it it tells you it's a graph, so you can both change text and make text accessible and if it's not, you can make the graphic itself accessible to a certain point. With that I will be turning it back over to John Gardner, thank you.
JOHN GARDNER: Okay, so we have sort of gone through several examples of what Iveo is, what it is all about. We had planned this webinar so we leave plenty of time for things to go wrong. And plenty of time for questions. And, this is the first time in my life I've ever given something like this wherenothing ever went wrong, and we through, we flew right. So I think what I will do is at least stop for a wild and throw the floor open for questions and comments and then I might show one or two more examples, but I think right now we have really said the major features of things that make Iveo so very nice.
NORM COOMBS: Okay this is Norm just with a couple instructions if you're wanting to try to use the mic to talk, use the control key, and don't keep it locked or you will keep everybody else, this is one person at a time, so please please please do not freeze the mic, and you can also put questions in the text went up. And if you do, either John Taylor or Beth will read them for John Gardner. Okay, who's got the first question?
: First question is what's a good minimum font size when you're reading text with the embosser.
JOHN GARDNER: The answer is it depends. If you are just reading some text in a string just about any font will do. These characters are big enough that I can almost not quite, but almost read it with my fingers because I don't know what the shape of the things is but typically text is quite a bit smaller and it just has, even if the text is way small compared to the resolution of the embosser which is 20 dots per inch, you cannot read it, but you can tell that it's text and if the text is there you can push on it and find it and make it work. The smaller the text of course the harder it is, the figures predict the text is pretty small and you may have to push around a couple times before you hit the actual plaintiff where the text is instead of hitting just below it etc. In something like one of these equations where you really want to read it character by character I would guess that these things are renting out with the equivalent of about 20 to 30, 20 to 20 points, maybe even some of them are even much bigger. We have one that is printing out as a bit about a 40 point and I can't read each character by itself. But the problem is at 20 points I can't really not read the characters by feeling their shape, but I can distinguish each character so I can push one character at a time and go through and read the equation, so that is sort of the font size that one needs on this graphic. But do not forget, when one gets a graphic, Iveo will allow you to, in fact it will automatically increase the size of the thing to full screen, so you made the F SCG in which the graphic was 3 x 4" and I imported it on something at which the default size was 14 inches it will expand the thing up until it fits as well as it can into something that is 11 x 14". So text that is pretty small on a 3 x 4 gets pretty big radio expanded of June 14, if I really wanted to read the text I could choose a rectangle around the text and expand it even more that's another of the features and I will show you that, shortly. The final text size needs to be in order to read individual characters probably 20 to 25.
NORM COOMBS: In case it is not clear to somebody, you don't have to be reading the text with your finger, you have to identify it, and then if you push the program under these will speak the
Next question says I guess they're just an understand at all what Iveo does how does it interface with the computer sorry, as a sighted person, I'm pretty confused.
JOHN GARDNER: Okay fair enough. It is interfacing with the computer by using a touchpad. This touchpad you can buy from you plus. And actually there are other types of electronic pad like a walk-on template that actually do work. This one is nice because it is such calibrated to use for this sort of thing but it's also relatively inexpensive. So, what I do is I put the graphic on to this touch tablet and it may need to be calibrated. The calibration what's done is usually pretty good, but things being what they are, they tend to drift a little bit, so to recalibrate it what I can do if I have the figure onscreen is to press a hotkey to recalibrate and asked me to touch the top left corner which I do and it asks me to touch the lower bottom corner which I do in that calibrates it better than it was calibrated already. But, what the purpose of the touchpad is to basically tell the computer where I graphic you are, just like the mouse on the screen tells you where you want to click for information, what you have is a tactile mouse basically, so I'm pressing this thing and the touchpad is calibrated to the screen so that it's equivalent to clicking. And in fact many people who are dyslexic for example do not use a touchpad at all, they simply use Iveo and they can simply use a touchscreen or a mouse and click on it and it works just fine so what it's doing is allowing me to hear objects and texts that are on the screen. That is the way that it's making it accessible. Is that clear?
Another question was, would this be used for math or science textbooks?
JOHN GARDNER: It certainly is used for math and science textbooks. If you are fortunate to have a way to separate out the graphics from the rest of the text and some PDFs allow you to do that, then you can simply, what am I trying to say import each of the graphics into Iveo and you will have a copy of every graphic in the book. Whatever list encouraged universities and academic institutions to do is to do what Kyle did for meal, send them an accessible SVG file because the best thing on what the book is, the graphic may be very poor attempt or it may be not very important and the student may not want to bother, just reading the caption on any graphic is quite enough, but if you do want to use it and let the student design which one they want, make the embosser available and make the touchpad available in the student lab, so the student can do them himself. The office has a relatively small amount of work to do on each graphic which means they can be a lot more productive so they can make a lot of graphics and instead of having to take them through beginning to end and make the tech tell everything, they just let the student choose which tactiles they want to make and emboss it out.
How does this work with the complex image like multiple colors or photos?
JOHN GARDNER: Okay, I will tell you what the embosser does with color graphics. It essentially converts the color image into what I call a tactile grayscale image. So colors are either light or dark. A dark color will emboss with big dots, a light color will emboss with small dots. We have variable.heights, so a light color will produce a sort of stippled pattern. If it is very light it will not emboss at all, the dark things emboss dark, it does not presently recognize color, it just recognizes grayscale. Grayscale is sometimes enough and sometimes not. So we are developing a patterning capabilities so that the user will have an option of turning on patterns so that instead of just having a grayscale image for patterns it will have patterns read will have a different pattern from blue or green I'm not sure if it will have artistic appreciation but will certainly help in doing things like color bar charts and pie charts in which sometimes the colors although they are different may be more or less the same darkness so it is a little hard to tell where one starts and the other one stops. Most color images, you can feel feel, let's just say it sort of bumps and wiggles, the bumps and wiggles by themselves really are really more or less useless but with the voice that goes with it becomes very useful indeed so you might feel sort of a bumpy line on something and you press it and maybe since we have the example of a heart, the heart is multicolored it is colored from dark red to light red and it's got veins and arteries running. And maybe you will touch on if these black arteries which is maybe only barely distinguishable from the dark red that is crossing at the present time, but you push it and it will say artery and you move off of it, and you the red part, will say (carteriod) on it, whatever it says, but, graphics in some instances can be pretty accessible in some sense is going to need either a human being to do some work on it to simplify, or what we hope relatively soon is we will have artificial intelligence that will allow the computer to do things like simplify the graphic and perhaps instead of showing all colors, it will show what is it called? An edge detection, so you will see the edges of the colors, this is a very complex subject there's a lot of work going on and we're going to try to take advantage of what has been done to give people some things to work on. What we're hoping is eventually most scientific type graphics will be reasonably accessible with minimal help from human beings and some of them are complicated things with colors particularly things that are dark red, dark blue, dark green may appear pretty much the same, so actually will need some human beings simplification. Thanks.
Anne wants to know, how do you make an accessible SVG file.
JOHN GARDNER: Okay, well John did it. What he did was to take a PDF and he imported it into the Iveo Creator Pro application, the import feature has OCR capability and to the extent that the bitmap text is reasonably well defined, the OCR works very well. And you will have the text text recognized and displayed as SVG text. He did show you that with a lot of graphics the text is not good enough to give a good OCR, so somebody has to overlay that, but what you are creating in the process of doing this is that accessible SVG. You can take, so you can import practically any electronic file format. And you can also simply open an SVG that's been made someplace else, like the Wolfram SVG and if you open it in Creator Pro and there are still things that are inaccessible, remember the equation that I showed you. A sighted person who wants to make that graphical little bit more accessible could just paste a rectangle on top of that and typing X plus BX plus C equals 13 and there you have it. But what you are doing is making an SVG and the purpose of the creator is not so much to allow you to create complex graphics. It will allow you to create some simple graphics, but will certainly import just about anything and allow you to make it accessible.
NEW SPEAKER: Okay, I'm trying to understand the whole process. You took the original graphic and you started with the Tiger printer. So I assume, you ended up with the braille representation of the shape, is that right?
JOHN GARDNER: Yes, actually you start with the SVG and you print the SVG under the Tiger embosser. This is what allows me and any blind person and perhaps some people who are severely dyslexic etc. to be able to recognize the graphic. So it is half way that I turned into a mass the other way is I have a touchpad that I put it on so the typographic, it is essential for me because that gives me a feel to coin a phrase, for the graphic and putting it allow on the text pad that allows me to pass a touchpad this feels like text, what does it say, this feels like an object, what is it?
NEW SPEAKER: That's what I assumed, I'm a blind mathematician actually. The touchpad you are using the touchpad just really to read whatever text is on the graphic, and then embossing it to get all the other attributes of it.
JOHN GARDNER: When I emboss it is emboss is the full graphic, the full text of the graphic, the touchpad is smooth, it has just a smooth surface so I can actually read it graphic without that overlay, but it is very tedious and very annoying because I have to feel around and find something until I touch it. I don't have very good spatial sense so that if I touch something and then I move away from it and come back to it I have difficulty doing that there are some blind people who I admire very much who can feel around on a perfectly flat surface there are people who use iPad for example, Ed Summers who I believe will be giving a talk next week, he can do a lot of things without any tactile graphic. He also permits one to use tactile graphics much as we are doing here for more complicated things.
Then the next question was, can graphics be interactive?
JOHN GARDNER: Yes, they can be interactive what I have showed you to this point is I guess what I would call smart, but not interactive graphics. What we've done is to show you how to read a static graphic. So, this would be a graphic for example something that appears in a paper book, or most figures that appear in electronic documents, some just static documents. SVG does have some animation does have ways of making things interactive and we have done a lot of exploratory work, we've begun developing curriculum materials for very young children in which there's a great deal of inactivity. Most of this we have done actually just using SVG, standard SVG occasionally we embellish it is a good putting stuff like JavaScript SVG, which is fine, it is still SVG, but to make interactive graphics takes a bit of authoring work. If you're interested in interactive graphics I would definitely suggest that you look at Ed Summers talk. He's got really beautiful interactive graphics that come from SAS database information.
Next question is what other file formats can import to Iveo?
JOHN GARDNER: I can't think of anything that will not important to Iveo one way or another. What John showed you was importing importing from PDF importing from bitmap images. We also have a pseudo-printer, which is for people who using psych Acrobat are familiar to you, so any application that prints like word for example if you have a word document you can simply open the Iveo converter, which comes as part of the Iveo package. Any text which is proper text will convert to SVG text and sometimes this is very nice sometimes the text that is not converted is not so nice because it is broken up a word will not pronounce as a word but as parts of words. This has to do with the way the application delivers the information to the printer driver. You always have the option of importing it in some way of making it the PDF if you want to use the scanner there's a standard input, I forgot, what is the name of the scan?
JOHN GARDNER: Anyway there is a standard scan interface that is not coming to my tongue at the moment, but it uses the port from just about any standard scanner, so you can scan something in a paper, you can take something with low image essentially anything.
Beth who was reading the text chat:
: That is all I have for questions I was going to make sure that viewplus.com was where they could go so they could visualize the process. But, Sheila already typed in link for us, so I'm out of questions here on the text box, I believe.
betsy speaking: Well, I have one off-the-wall thought. Can I be off-the-wall a minute?
JOHN GARDNER: Sure, be off any wall you want.
You are talking about a string that would read mathematics and I wonder if there would be any way to use (inaudible) and have a translator in there. Because there are some scanning programs that would read, you know, translate that and read it correctly.
JOHN GARDNER: Of course, you can use tech, we have chosen to use math ML because math ML is a little more modern and it's pretty straightforward to convert Logitech equations into math ML. There are some people who would like for us to put in LA tech, in fact if we develop this capability what we will probably do is to put in both. Right now I have a pretty good application that speaks math ML, and (inaudible) etc. and I don't have an easy one for LA tech but of course you concerned into that and that could be a user option.
NEW SPEAKER: Math ML is certainly preferred I was thinking of a single way to get a string that's all. We need to talk because there's a lot of work, I'm doing some piece of work for my own work, nothing like you are doing, but you are expanding on some things that I use that I think would be very helpful, thank you.
NORM COOMBS: John, that is Betsy Dunn from Connecticut, as I recall. And I will get her in touch with you and vice versa in an e-mail through me.
JOHN GARDNER: Great. If you read the announcement from them you may have noticed that I'm giving another webinar in two weeks being a glutton for punishment and the topic of that webinar is called lean mass. Is I suppose it is a notationbut what I will show you is an editor that I've developed over the last year or so presently I'm using as an interface to math type in Word so I'll just give you a little flavor of what you are going to see. We have working pretty well now the ability to convert a Word document so that the MathType equations display as lean which is simply a string of Unicode characters and if you're screenwriter has a dictionary put into it and those are available if you just want to put it in it will read the equation to you in a very compact, intuitive format. For example it will say fraction 3+7/1+ X infraction etc. etc. That is nice and you can hit another button and sent it back to MathType conversion, but the very nice thing is that you can, not just that ability, is perhaps useful, but there are other ways of reading math and MathType is becoming accessible by itself now, the really nice thing is that you can open the equations in the lean editor and edit them, you can also create the equations and insert them to words of a blind person using the editor can do math. I've put out a lot of nice function so that you don't just write math, reading and writing Matthew can actually manipulate maps so there are some really nice things you can copy and paste fractions, convert fractions convert strings to fractions and convert fractions to strings and cut out the denominator etc. all with a single keystroke so there are also menus for you to do these things so you don't have to member that keystrokes if you don't want to, but if you are a power user, this makes manipulating equations much easier than it once was the just doing this now it's built to be an audio browser because there is no braille for the special Unicode characters. There is because I've developed a braille, I will not call her braille code because it's not really a code, it is simply a representation of each one of these characters. And you are welcome to use it. It will display in braille, you can also change the braille to something else that you want, and it will also it doesn't do this today, but it will eventually also display the equation in standard math codes like for example image, one of the things that I do not know how you do is to write this kind of an editor, so you can actually just type in MS and edited so that images actually check showing you the equation but you are actually editing it in the lean notation if you edit and insert or change something, then it compiles the Nimitz, so you can see what you have done in Nimitz so that may not be working in two weeks, it probably will not but it's relatively straightforward to do that and it will be done fairly soon. So, for people who can tolerate doing what I will call audio math, and reading it in braille as well as I think you're going to really like this application.
NORM COOMBS: Talking about punishment, John it is the captioner that I really feel sympathy for but I will take what she gives me and try to clean it up a little and put it into a little bit better shape, but this is a pretty hard Webinar to turn into transcription but she's a good one and between what she gives me and my attempted clean it up I think we will have something in two or three days especially as my Denver daughter and granddaughter are coming for a few days, so that will keep me occupied for a few days. So I think we are just about at the end of the hour let's give them ended in case someone wants to put a last-minute questions in the text window or grab the mic.
JOHN GARDNER: Can we get an e-mail for John?
JOHN GARDNER: Sure you can, it's pretty easy, it is John.Gardner@viewplus.com. View plus is spelled VI EW PL US.com and I hope that the transcriptionist can type that in and I apologize for the jargon that I was using last time. Norm asked me not to do that and I forgot.
NORM COOMBS: When I put the audio it will have links to maybe the view plus side as well so I want to thank John, for being brave and tackling desktop sharing, and not a complex topic. I want to thank everybody who came and stayed, and I'm not a mathematician and I do not want to learn much of this, I'm really excited to see the kind of progress that is being made not just for the presentation but also for his work but trying to make science and math more accessible. So, thank you John Gardner, John Taylor, Beth, Shouldest and everyone else. And with that I will close the recording.