{"id":71,"date":"2022-10-18T22:00:05","date_gmt":"2022-10-18T22:00:05","guid":{"rendered":"http:\/\/www.kerneltrick.com\/?p=71"},"modified":"2022-10-18T22:00:05","modified_gmt":"2022-10-18T22:00:05","slug":"handwritten-character-recognition-neural-net","status":"publish","type":"post","link":"https:\/\/www.kerneltrick.com\/?p=71","title":{"rendered":"Handwritten Character recognition (Neural Net)"},"content":{"rendered":"\n<p>Here is a program that demonstrates using a neural network to recognize handwritten characters.<br>Not as verbose as my other entries as I wanted to keep it under 200 lines.<br>Written in BMaxNG.<br>You&#8217;ll need to download the training file <a href=\"https:\/\/www.kerneltrick.com\/download\/optdigits.tra\">here<\/a> .<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><a href=\"https:\/\/www.kerneltrick.com\/wp-content\/uploads\/2022\/10\/digirec.png\"><img loading=\"lazy\" decoding=\"async\" width=\"506\" height=\"523\" src=\"https:\/\/www.kerneltrick.com\/wp-content\/uploads\/2022\/10\/digirec.png\" alt=\"\" class=\"wp-image-72\" srcset=\"https:\/\/www.kerneltrick.com\/wp-content\/uploads\/2022\/10\/digirec.png 506w, https:\/\/www.kerneltrick.com\/wp-content\/uploads\/2022\/10\/digirec-290x300.png 290w\" sizes=\"auto, (max-width: 506px) 100vw, 506px\" \/><\/a><\/figure>\n\n\n\n<pre title=\"Source Code (BlitzmaxNG)\" class=\"wp-block-code has-small-font-size\"><code lang=\"visual-basic\" class=\"language-visual-basic line-numbers\">SuperStrict\nFramework brl.basic\nImport brl.glmax2d\nImport brl.retro\nImport brl.collections\nAppTitle = \"Handwritten Digit Recognition\"\n\nGraphics 500, 500\nSeedRnd(MilliSecs())\n\nGlobal x#[][]\nGlobal y#[][]\nLoadData()\n\nGlobal NumBF% = 1 + Sqr(x.Length)\nGlobal bf#[][]  = New Float[NumBF][]\nGlobal W#[,] = New Float[NumBF, 10]\nGlobal Out#[] = New Float[10]\nGlobal Sigma# = 2\nGlobal rout#[Numbf]\nGlobal err#\nGlobal trainitr%\nGlobal Kern#(a#[], b#[], s#) = RKernel 'RKernel 'IQuadKernel\n\n\n'\/\/ INIT \/\/\nFor Local i% = 0 Until NumBF\n\tbf[i] = x[Rand(x.Length - 1)][..]\n\tFor Local j% = 0 Until 10\n\t\tW[i, j] = Rnd(1 \/ Sqr(64)) * Sgn(Rnd(-1, 1))\n\tNext\nNext\n\nWhile Not KeyHit(KEY_ESCAPE)\n\tCls\n\t\tDrawData()\n\t\tLocal err# = (Train(x, y))\n\t\tIf(trainitr &lt; 800) Then DrawText(\"Training\" +[\".\", \"..\", \"...\"][trainitr Mod 3], 400, 10)\n\tFlip\nWend\n\n'----------------------------------------------------------------------------\nFunction Process%(in#[])\n\tMemClear(VarPtr Out[0], Out.Length * SizeOf(1:Float))\n\tFor Local j% = 0 Until numBF\n\t\tLocal x# = kern(bf[j], in, Sigma)\n\t\tFor Local k% = 0 Until Out.Length\n\t\t\tOut[k]:+(w[j, k] * x)\n\t\tNext\n\t\trout[j] = x\n\tNext\n\t\n\tFor Local k% = 0 Until Out.Length\n\t\tOut[k] = sig(Out[k])\n\t\terr:+Sqr((in[k] - Out[k]) ^ 2)\n\tNext\nEnd Function\n'----------------------------------------------------------------------------\nFunction Train#(TData#[][], Targ#[][])\n\tIf trainitr > 800 Then Return 0\n\n\tFor Local i% = 0 Until TData.Length\n\t\terr = 0\n\t\tLocal idx% = i\n\t\tLocal t#[] = targ[idx]\n\t\tProcess(TData[i])\n\t\tFor Local j% = 0 Until NumBF\n\t\t\tFor Local k% = 0 Until Out.Length\n\t\t\t\tLocal d# = (t[k] - Out[k])\n\t\t\t\tw[j, k]:+(0.01 * d * rout[j])\n\t\t\tNext\n\t\tNext\n\tNext\n\ttrainitr:+1\n\tIf trainitr &lt; 300 Then lvq()\n\tReturn err\nEnd Function\n'----------------------------------------------------------------------------\nFunction lvq()\n\tLocal dat#[] = x[Rand(x.Length - 1)]\n\tLocal d# = 99999\n\tLocal b%\n\tFor Local i% = 0 Until bf.Length\n\t\tLocal td# = aDist(dat, bf[i])\n\t\tIf(td) &lt; d Then\n\t\t\td = td\n\t\t\tb = i\n\t\tEndIf\n\tNext\n\tFor Local i% = 0 Until bf[b].Length\n\t\tbf[b][i]:+((0.1) * (dat[i] - bf[b][i]))\n\tNext\nEnd Function\n'----------------------------------------------------------------------------\nFunction sig#(x#) Inline\n\tReturn 8.06356 \/ (0.422614 * x + 8.21913 \/ x + 16.2612 \/ (0.43677 * x + 7.87788 \/ x))  't\nEnd Function\n'----------------------------------------------------------------------------\nFunction Normalize#(val#, desmin#, desmax#, natmin#, natmax#) Inline\n\tReturn desmin + (val-natmin)*(desmax-desmin)\/(natmax-natmin)\nEnd Function\n'----------------------------------------------------------------------------\nFunction aDist:Float(v1:Float[], v2:Float[])\n\tLocal val:float = 0\n\tFor Local i:Int = 0 Until v1.Length\n\t\tval:+((v1[i] - v2[i]) * (v1[i] - v2[i]))\n\tNext\n\tReturn Sqr(val)\nEnd Function\n'----------------------------------------------------------------------------\nFunction IQuadKernel:Float(vA:Float[], vB:Float[], Sigma:Float)\n\tLocal prod#=0\n\tFor Local i% = 0 Until va.length\n\t\tprod :+ ((va[i] - vb[i]) * (va[i] - vb[i]))\n\tNext\n\tprod = 1\/(1+prod\/(sigma*sigma))  'Inv quad\n\tReturn prod\nEnd Function\n'----------------------------------------------------------------------------\nFunction RKernel:Float(vA:Float[], vB:Float[], Sigma:Float)\n    Local n#\n    For Local i% = 0 Until va.length\n\t\tn:+(vA[i] - vB[i]) * (vA[i] - vB[i])\n    Next\n    Local d# = 2.0 * Sigma * Sigma\n    Local z# = n \/ d\n    Return Exp(-z)\nEnd Function\n'----------------------------------------------------------------------------\nFunction DrawData()\n\tLocal t% = Max(0, Abs(MilliSecs() \/ 3000))\n\tLocal i% = t Mod x.Length - 1\n\t\tSetColor 255, 255, 255\n\t\tLocal Count%\n\t\tLocal c#\n\t\tFor Local yy# = 0 Until 8\n\t\t\tFor Local xx# = 0 Until 8\n\t\t\t\tc = Normalize(x[i][Count], 0, 255, -1, 1)\n\t\t\t\tSetColor(c, c, c)\n\t\t\t\tDrawRect(100 + xx * 40, 50 + yy * 50, 40, 50)\n\t\t\t\tCount:+1\n\t\t\tNext\n\t\tNext\n\tSetColor 255, 255, 255\n\tFor Local j% = 0 Until y[i].Length\n\t\tIf y[i][j] > 0.5 Then DrawText(\"True: \" + j, 10, 480)\n\tNext\n\t\n\tProcess(x[i])\n\n\tLocal hi%, hv# = -999\n\tLocal tn%\n\tFor Local j% = 0 Until 10\n\t\tIf Out[j] > hv Then hi = j;hv = Out[j]\n\t\tIf y[i][j] > 0.5 Then tn = j\n\tNext\n\t\n\tFor Local j% = 0 Until 10\n\t\tSetColor 255, 255, 255\n\t\tIf j = hi And hi &lt;> tn Then SetColor 255, 0, 0  'Else SetColor 255, 255, 255\n\t\tIf j = hi Then SetColor 0, 255, 0 Else SetColor 255, 255, 255\n\t\tDrawText(j + \": \" + Int(Max(Out[j], 0) * 100) + \"%\", 10, 40 + (j * 20))\n\tNext\n\tDrawText(\"Guess:\", 10, 10)\nEnd Function\n'----------------------------------------------------------------------------\nFunction LoadData()\n\tLocal dlist:TLinkedList &lt; Float[] >= New TLinkedList &lt; Float[] > ()\n\tLocal ylist:TLinkedList &lt; Float[] >= New TLinkedList &lt; Float[] > ()\n\tLocal F$ = LoadText(\"optdigits.tra\")\n\tIf Trim(F).Length = 0 Then RuntimeError(\"Oops!\")\n\tLocal line$[] = f.split(\"~n\")\n\tFor Local i% = 0 Until line.length\n\t\tIf Rnd(0,1)>0.3 Then Continue\n\t\tLocal aline$[] = LINE[i].Split(\",\")\n\t\tLocal tmpy#[10]\n\t\ttmpy[Int(aline[64]) ] = 1\t\t\n\t\tLocal tmpX#[64]\n\t\tFor Local j% = 0 Until 64\n\t\t\ttmpX[j] = (Float(aline[j]) - 7) \/ 8\n\t\tNext\n\t\tdlist.AddLast(tmpX)\n\t\tylist.AddLast(tmpy)\n\tNext\n\tx = dlist.ToArray()\n\ty = ylist.ToArray()\nEnd Function<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Here is a program that demonstrates using a neural network to recognize handwritten characters.Not as verbose as my other entries as I wanted to keep it under 200 lines.Written in BMaxNG.You&#8217;ll need to download the training file here . SuperStrict Framework brl.basic Import brl.glmax2d Import brl.retro Import brl.collections AppTitle = &#8220;Handwritten Digit Recognition&#8221; Graphics 500,&#8230; <\/p>\n<div class=\"read-more navbutton\"><a href=\"https:\/\/www.kerneltrick.com\/?p=71\">Read More<i class=\"fa fa-angle-double-right\"><\/i><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-71","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=\/wp\/v2\/posts\/71","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=71"}],"version-history":[{"count":0,"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=\/wp\/v2\/posts\/71\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=71"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=71"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kerneltrick.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=71"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}