{"id":59505,"date":"2020-10-05T10:17:11","date_gmt":"2020-10-05T01:17:11","guid":{"rendered":"https:\/\/smilegate.ai\/?p=59505"},"modified":"2020-10-05T10:22:24","modified_gmt":"2020-10-05T01:22:24","slug":"huggingface-onnx-runtime","status":"publish","type":"post","link":"https:\/\/smilegate.ai\/cn\/2020\/10\/05\/huggingface-onnx-runtime\/","title":{"rendered":"\u4f7f\u7528HuggingFace\u548cONNX Runtime\u8fdb\u884cNLP\u52a0\u901f"},"content":{"rendered":"

\u57fa\u4e8eTransformer\u7684\u8bed\u8a00\u6a21\u578b\u663e\u793a\u51fa\u7684\u6027\u80fd\u6539\u8fdb\u4ee4\u4eba\u60ca\u8bb6\uff0c\u4f46\u662f\u968f\u7740\u6a21\u578b\u5927\u5c0f\u7684\u6210\u500d\u589e\u957f\uff0c\u5bf9\u670d\u52a1\u6210\u672c\u7684\u5173\u6ce8\u4e5f\u53d8\u5f97\u8d8a\u6765\u8d8a\u91cd\u8981\u3002\u5bf9\u4e8e\u57fa\u4e8eBert\u6216GPT-2\u7684\u60c5\u51b5\uff0c\u5927\u7ea6\u67091\u4ebf\u4e2a\u53c2\u6570\uff0c\u56e0\u6b64\u5fc5\u987b\u5728\u5f88\u5927\u7a0b\u5ea6\u4e0a\u4f18\u5316\u6a21\u578b\u5927\u5c0f\uff0c\u5185\u5b58\u5e26\u5bbd\u548c\u63a8\u65ad\u65f6\u95f4\u3002<\/p>\n\n\n\n

\u7528\u4e8e\u6a21\u578b\u4f18\u5316\u7684\u4e24\u79cd\u4ee3\u8868\u6027\u6280\u672f\u662f\u84b8\u998f\u548c\u91cf\u5316\u3002\u5171\u4eab\u94fe\u63a5\u662fHuggingface\u548cMicrosoft\u5de5\u4f5c\u7684\u7ed3\u679c\uff0c\u5c06INT8\u91cf\u5316\u548cONNX\u8fd0\u884c\u65f6\u5e94\u7528\u4e8eHuggingface\u7684\u6a21\u578b\uff0c\u5e76\u5206\u6790\u4e86\u6700\u65b0CPU\u652f\u6301\u7684SIMD\u6307\u4ee4\u96c6\uff08AVX2\uff0cAVX512 VNNI\uff09\u7684\u6027\u80fd\u3002\u4f5c\u4e3a\u53c2\u8003\uff0cAVX2\u5177\u6709256\u4f4d\u5bc4\u5b58\u5668\uff0cAVX512\u652f\u6301512\u4f4d\u5bc4\u5b58\u5668\uff0c\u4ee5\u53ca\u4e00\u4e2a\u79f0\u4e3aVNNI\u7684CNN\u7279\u5b9a\u6307\u4ee4\u96c6\u3002<\/p>\n\n\n\n

\u603b\u800c\u8a00\u4e4b\uff0c\u5e94\u7528INT8\u91cf\u5316\u548cONNX\u8fd0\u884c\u65f6\u53ef\u4ee5\u5c06\u6a21\u578b\u5927\u5c0f\u51cf\u5c0f\u5230\u5927\u7ea6\u56db\u5206\u4e4b\u4e00\uff0c\u800c\u6027\u80fd\u51e0\u4e4e\u6ca1\u6709\u4e0b\u964d\u3002\u6b64\u5916\uff0cAVX2\u7684\u63a8\u7406\u901f\u5ea6\u4e5f\u63d0\u9ad8\u4e861.6\u500d\uff0cAVX512 VNNI\u7684\u63a8\u7406\u901f\u5ea6\u4e5f\u63d0\u9ad8\u4e863\u500d\u4ee5\u4e0a\u3002\u5176\u4e2d\uff0cINT8\u91cf\u5316\u7684\u8d21\u732e\u7ea6\u4e3a85%\uff0c\u4e0ePytorch\u76f8\u6bd4\uff0c\u901a\u8fc7\u5e94\u7528ONNX\u8fd0\u884c\u65f6\u6240\u83b7\u5f97\u7684\u901f\u5ea6\u63d0\u9ad8\u4e86\u7ea615%\u3002<\/p>\n\n\n\n

\u53e6\u4e00\u65b9\u9762\uff0c\u5982\u679c\u6d4b\u91cf\u63a8\u65ad\u65f6\u95f4\uff0c\u5219\u57fa\u4e8e\u6279\u6b21\u5927\u5c0f1\uff0c\u5e8f\u5217\u957f\u5ea6128\uff0cGPT-2\u548cAVX512 VNNI\uff0cPytorch FP32\u7ea6\u4e3a58ms\uff1b\u5e94\u7528ONNX\u8fd0\u884c\u65f6\u65f6\u4e3a45ms\uff0c\u5e94\u7528INT8\u91cf\u5316\u65f6\u4e3a20ms\u3002\u8003\u8651\u5230\u4ee5\u4e0a\u7ed3\u679c\u4e0d\u9002\u7528\u4e8e\u5176\u4ed6\u4f18\u5316\u6280\u672f\uff0c\u4f8b\u5982\u84b8\u998f\uff0c\u56e0\u6b64\u6211\u8ba4\u4e3aINT8\u91cf\u5316\u548cONNX\u8fd0\u884c\u65f6\u5e94\u7528\u7a0b\u5e8f\u90fd\u662f\u670d\u52a1\u5316\u5fc5\u4e0d\u53ef\u5c11\u7684\u9879\u76ee\u3002<\/p>\n\n\n\n

\u5728\u6df1\u5ea6\u5b66\u4e60\u5b66\u4e60\u5e02\u573a\u4e2d\uff0cGPU\u662f\u5fc5\u4e0d\u53ef\u5c11\u7684\uff0c\u4e0d\u53ef\u80fd\u7528CPU\u53d6\u4ee3\u5b83\u4eec\uff0c\u4f46\u662f\u63a8\u7406\u5e02\u573a\u5c1a\u672a\u5f97\u51fa\u7ed3\u8bba\u3002 NVidia Ampere\u591a\u5b9e\u4f8bGPU\uff0cIntel AVX512 VNNI\uff0cGoogle TPU\uff0c\u9ad8\u901a\u795e\u7ecf\u5904\u7406\u5f15\u64ce\uff0c\u534e\u4e3a\u9e92\u9e9f\uff0cApple Bionic\u7b49\uff0c\u90fd\u53ef\u4ee5\u770b\u4f5c\u662f\u4ed6\u4eec\u52aa\u529b\u62e5\u6709\u81ea\u5df1\u7684\u89e3\u51b3\u65b9\u6848\u7684\u4e00\u6b65\u3002\u5728\u8fd9\u65b9\u9762\uff0c\u6765\u81ea\u4e0d\u540c\u4f9b\u5e94\u5546\u7684\u4ea7\u54c1\u7684\u4e92\u64cd\u4f5c\u6027\u548c\u6807\u51c6\u5316\u53ef\u80fd\u5f88\u91cd\u8981\uff0c\u6211\u8ba4\u4e3a\u6a21\u578b\u8868\u8fbe\u6807\u51c6\u5316\u6846\u67b6\uff08\u4f8b\u5982ONNX\uff09\u5c06\u5728\u8fd9\u65b9\u9762\u505a\u51fa\u5f88\u5927\u8d21\u732e\u3002\u4ee5\u4e0b\u662f\u4e00\u8d77\u4f7f\u7528HuggingFace\u548cONNX\u8fd0\u884c\u65f6\u7684\u4ecb\u7ecd\u548c\u5b9e\u9a8c\u7ed3\u679c\u3002<\/p>\n\n\n\n

<\/div>\n\n\n
<\/a>
\"\"<\/div><\/div>
Hugging Face\u548cONNX Runtime\u53ef\u4ee5\u5b9e\u73b0\u66f4\u5feb\uff0c\u66f4\u5c0f\u7684\u91cf\u5316NLP<\/div>
\u91cf\u5316\u548c\u84b8\u998f\u662f\u7528\u4e8e\u5904\u7406\u6a21\u578b\u5927\u5c0f\u548c\u6027\u80fd\u6311\u6218\u7684\u4e24\u79cd\u5e38\u7528\u6280\u672f\u3002\u6211\u4eec\u8ba8\u8bba\u4e86\u53ef\u4ee5\u8f7b\u677e\u5730\u5e94\u7528\u4e8e\u6a21\u578b\u800c\u65e0\u9700\u91cd\u65b0\u8bad\u7ec3\u7684\u91cf\u5316\u3002<\/div><\/div><\/div>\n\n\n
<\/div>\n\n\n\n

\u6211\u4eec\u8fd8\u5171\u4eab\u4e86\u4e00\u4e2a\u6307\u5411HuggingFace\u8f6c\u6362\u5668\u548cONNX Runtime github\u7684\u94fe\u63a5\u3002<\/p>\n\n\n\n

<\/div>\n\n\n
<\/a>
\"\"<\/div><\/div>
\u62e5\u62b1\u8138\/\u53d8\u5f62\u91d1\u521a<\/div>
\ud83e\udd17\u53d8\u5f62\u91d1\u521a\uff1aPytorch\u548cTensorFlow 2.0\u7684\u6700\u65b0\u81ea\u7136\u8bed\u8a00\u5904\u7406\u3002 \u2013\u62e5\u62b1\u9762\/\u53d8\u5f62\u91d1\u521a<\/div><\/div><\/div>\n\n
<\/a>
\"\"<\/div><\/div>
\u5fae\u8f6f\/ onnxruntime<\/div>
ONNX\u8fd0\u884c\u65f6\uff1a\u8de8\u5e73\u53f0\uff0c\u9ad8\u6027\u80fdML\u63a8\u7406\u548c\u57f9\u8bad\u52a0\u901f\u5668\u2013 Microsoft \/ onnxruntime<\/div><\/div><\/div>
<\/span><\/div>","protected":false},"excerpt":{"rendered":"

Transformer \uae30\ubc18\uc758 \uc5b8\uc5b4 \ubaa8\ub378\ub4e4\uc774 \ubcf4\uc5ec\uc8fc\ub294 \uc131\ub2a5\uc801 \ud5a5\uc0c1\uc740 \ub180\ub78d\uc9c0\ub9cc \ubaa8\ub378 \ud06c\uae30\uac00 \uae30\ud558\uae09\uc218\uc801\uc73c\ub85c \ub298\uc5b4\ub098\uba74\uc11c \uc11c\ube44\uc2a4 \ube44\uc6a9\uc5d0 \ub300\ud55c \uace0\ubbfc\ub3c4 \uc911\uc694\ud574\uc9c0\uace0 \uc788\uc2b5\ub2c8\ub2e4. Bert-base\ub098 GPT-2\uc758 \uacbd\uc6b0 \uc57d 1\uc5b5\uac1c\uc758 \ud30c\ub77c\ubbf8\ud130\ub97c \uac00\uc9c0\uace0 \uc788\uc5b4\uc11c \ubaa8\ub378 \ud06c\uae30, \uba54\ubaa8\ub9ac \ub300\uc5ed\ud3ed, \ucd94\ub860 \uc2dc\uac04 \ubaa8\ub450 \uc0c1\ub2f9\ud55c \uc218\uc900\uc758 \ucd5c\uc801\ud654\uac00 \uc774\ub8e8\uc5b4\uc838\uc57c\ub9cc \ud569\ub2c8\ub2e4. \ubaa8\ub378 \ucd5c\uc801\ud654\uc5d0 \uc0ac\uc6a9\ub418\ub294 \ub300\ud45c\uc801\uc778 \uae30\uc220 \ub450 \uac00\uc9c0\ub294 distillation\uacfc quantization\uc785\ub2c8\ub2e4. \uacf5\uc720 \ub9c1\ud06c\ub294 Huggingface\uc640 Microsoft\uac00 \ud568\uaed8 \uc791\uc5c5\ud55c \uacb0\uacfc\ubb3c\ub85c\uc11c, INT8…<\/p>\n

<\/span><\/div>","protected":false},"author":1,"featured_media":59513,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rank_math_lock_modified_date":false,"footnotes":""},"categories":[18,196],"tags":[188,343,344,345],"class_list":["post-59505","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech03","category-code","tag-featured","tag-hugging-face","tag-onnx-runtime","tag-optimization","category-18","category-196","description-off"],"_links":{"self":[{"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/posts\/59505","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/comments?post=59505"}],"version-history":[{"count":3,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/posts\/59505\/revisions"}],"predecessor-version":[{"id":59511,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/posts\/59505\/revisions\/59511"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/media\/59513"}],"wp:attachment":[{"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/media?parent=59505"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/categories?post=59505"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/smilegate.ai\/cn\/wp-json\/wp\/v2\/tags?post=59505"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}