{"id":126117,"date":"2024-03-24T21:22:06","date_gmt":"2024-03-24T12:22:06","guid":{"rendered":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/"},"modified":"2024-04-05T09:31:23","modified_gmt":"2024-04-05T00:31:23","slug":"java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9","status":"publish","type":"post","link":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/","title":{"rendered":"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9"},"content":{"rendered":"<p>Java\u306eTokenStream\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u3092\u89e3\u6790\u3059\u308b\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u5165\u529b\u30c6\u30ad\u30b9\u30c8\u3092\u30c8\u30fc\u30af\u30f3\u306b\u5206\u89e3\u3057\u307e\u3059\u3002\u5404\u30c8\u30fc\u30af\u30f3\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u5185\u306e\u5358\u8a9e\u307e\u305f\u306f\u8a18\u53f7\u3092\u8868\u3057\u307e\u3059\u3002TokenStream\u3092\u4f7f\u7528\u3059\u308b\u305f\u3081\u306e\u624b\u9806\u306f\u6b21\u306e\u3068\u304a\u308a\u3067\u3059\u3002<\/p>\n<ol>\n<li>\u5fc5\u8981\u306a\u30af\u30e9\u30b9\u3092\u30a4\u30f3\u30dd\u30fc\u30c8\u3057\u307e\u3059\u3002<\/li>\n<\/ol>\n<pre class=\"post-pre\"><code><span class=\"hljs-keyword\">import<\/span> org.apache.lucene.analysis.TokenStream;\r\n<span class=\"hljs-keyword\">import<\/span> org.apache.lucene.analysis.Tokenizer;\r\n<span class=\"hljs-keyword\">import<\/span> org.apache.lucene.analysis.standard.StandardAnalyzer;\r\n<span class=\"hljs-keyword\">import<\/span> org.apache.lucene.analysis.tokenattributes.CharTermAttribute;\r\n<\/code><\/pre>\n<ol>\n<li>\u6a19\u6e96\u7684\u306a\u30a2\u30ca\u30e9\u30a4\u30b6\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u3092\u4f5c\u6210\u3059\u308b\uff1a<\/li>\n<\/ol>\n<pre class=\"post-pre\"><code><span class=\"hljs-type\">StandardAnalyzer<\/span> <span class=\"hljs-variable\">analyzer<\/span> <span class=\"hljs-operator\">=<\/span> <span class=\"hljs-keyword\">new<\/span> <span class=\"hljs-title class_\">StandardAnalyzer<\/span>();\r\n<\/code><\/pre>\n<ol>\n<li>\u30c8\u30fc\u30af\u30ca\u30a4\u30b6\u30fc\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u3092\u4f5c\u6210\u3057\u3066\u3001\u5206\u6790\u3059\u308b\u30c6\u30ad\u30b9\u30c8\u3092\u5165\u529b\u3057\u307e\u3059\u3002<\/li>\n<\/ol>\n<pre class=\"post-pre\"><code><span class=\"hljs-type\">Tokenizer<\/span> <span class=\"hljs-variable\">tokenizer<\/span> <span class=\"hljs-operator\">=<\/span> analyzer.tokenizer();\r\ntokenizer.setReader(<span class=\"hljs-keyword\">new<\/span> <span class=\"hljs-title class_\">StringReader<\/span>(<span class=\"hljs-string\">\"This is a sample text.\"<\/span>));\r\n<\/code><\/pre>\n<ol>\n<li>TokenStream\u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u3092\u53d6\u5f97\u3059\u308b\uff1a<\/li>\n<\/ol>\n<pre class=\"post-pre\"><code><span class=\"hljs-type\">TokenStream<\/span> <span class=\"hljs-variable\">tokenStream<\/span> <span class=\"hljs-operator\">=<\/span> tokenizer;\r\n<\/code><\/pre>\n<ol>\n<li>TokenStream\u5185\u306e\u30c8\u30fc\u30af\u30f3\u3092\u9806\u756a\u306b\u8fbf\u3063\u3066\u3001\u305d\u306e\u30c6\u30ad\u30b9\u30c8\u8868\u73fe\u3092\u51fa\u529b\u3059\u308b<\/li>\n<\/ol>\n<pre class=\"post-pre\"><code><span class=\"hljs-type\">CharTermAttribute<\/span> <span class=\"hljs-variable\">termAttribute<\/span> <span class=\"hljs-operator\">=<\/span> tokenStream.addAttribute(CharTermAttribute.class);\r\ntokenStream.reset(); <span class=\"hljs-comment\">\/\/ \u91cd\u7f6eTokenStream<\/span>\r\n<span class=\"hljs-keyword\">while<\/span> (tokenStream.incrementToken()) {\r\nSystem.out.println(termAttribute.toString());\r\n}\r\ntokenStream.end(); <span class=\"hljs-comment\">\/\/ \u7ed3\u675fTokenStream<\/span>\r\ntokenStream.close(); <span class=\"hljs-comment\">\/\/ \u5173\u95edTokenStream<\/span>\r\n<\/code><\/pre>\n<p>\u4e0a\u8a18\u306e\u30b3\u30fc\u30c9\u3067\u306f\u3001StandardAnalyzer \u304c\u30c6\u30ad\u30b9\u30c8\u3092\u30c8\u30fc\u30af\u30f3\u306b\u5206\u3051\u3066\u3001TokenStream \u30aa\u30d6\u30b8\u30a7\u30af\u30c8\u3092\u8fd4\u3057\u307e\u3059\u3002tokenStream.incrementToken() \u30e1\u30bd\u30c3\u30c9\u3092\u547c\u3073\u51fa\u3059\u3053\u3068\u3067\u3001TokenStream \u5185\u306e\u30c8\u30fc\u30af\u30f3\u3092 1 \u3064\u305a\u3064\u53d6\u5f97\u3067\u304d\u307e\u3059\u3002tokenStream.addAttribute() \u30e1\u30bd\u30c3\u30c9\u3092\u4f7f\u7528\u3059\u308b\u3068\u3001\u30c8\u30fc\u30af\u30f3\u306e\u30c6\u30ad\u30b9\u30c8\u8868\u73fe\u3092\u53d6\u5f97\u3067\u304d\u307e\u3059\u3002\u6700\u5f8c\u306b\u3001tokenStream.end() \u30e1\u30bd\u30c3\u30c9\u3068 tokenStream.close() \u30e1\u30bd\u30c3\u30c9\u3092\u547c\u3073\u51fa\u3057\u3066\u3001TokenStream \u3092\u7d42\u4e86\u3057\u3066\u9589\u3058\u307e\u3059\u3002<\/p>\n<p>\u4e0a\u8a18\u306e\u30b3\u30fc\u30c9\u3067\u6a19\u6e96\u7684\u306aLucene\u306e\u30a2\u30ca\u30e9\u30a4\u30b6\u30fc\u3092\u4f7f\u7528\u3057\u3066\u3044\u308b\u70b9\u306b\u6ce8\u610f\u3057\u3066\u304f\u3060\u3055\u3044\u3002\u5225\u306e\u30a2\u30ca\u30e9\u30a4\u30b6\u30fc\u3092\u4f7f\u7528\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b\u5834\u5408\u306f\u3001\u5fc5\u8981\u306b\u5fdc\u3058\u3066\u7f6e\u304d\u63db\u3048\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Java\u306eTokenStream\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u3092\u89e3\u6790\u3059\u308b\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u5165\u529b\u30c6\u30ad\u30b9\u30c8\u3092\u30c8\u30fc\u30af\u30f3\u306b\u5206\u89e3\u3057\u307e\u3059\u3002\u5404\u30c8\u30fc\u30af\u30f3\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u5185\u306e\u5358\u8a9e\u307e\u305f\u306f\u8a18\u53f7\u3092\u8868\u3057\u307e\u3059\u3002TokenStream\u3092\u4f7f\u7528\u3059\u308b\u305f\u3081\u306e\u624b\u9806\u306f\u6b21\u306e\u3068\u304a\u308a\u3067\u3059\u3002 [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-126117","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v21.5 (Yoast SEO v21.5) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9 - Blog - Silicon Cloud<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.silicloud.com\/ja\/blog\/java\u306etokenstream\u306e\u4f7f\u3044\u65b9\/\" \/>\n<meta property=\"og:locale\" content=\"ja_JP\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9\" \/>\n<meta property=\"og:description\" content=\"Java\u306eTokenStream\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u3092\u89e3\u6790\u3059\u308b\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u5165\u529b\u30c6\u30ad\u30b9\u30c8\u3092\u30c8\u30fc\u30af\u30f3\u306b\u5206\u89e3\u3057\u307e\u3059\u3002\u5404\u30c8\u30fc\u30af\u30f3\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u5185\u306e\u5358\u8a9e\u307e\u305f\u306f\u8a18\u53f7\u3092\u8868\u3057\u307e\u3059\u3002TokenStream\u3092\u4f7f\u7528\u3059\u308b\u305f\u3081\u306e\u624b\u9806\u306f\u6b21\u306e\u3068\u304a\u308a\u3067\u3059\u3002 [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.silicloud.com\/ja\/blog\/java\u306etokenstream\u306e\u4f7f\u3044\u65b9\/\" \/>\n<meta property=\"og:site_name\" content=\"Blog - Silicon Cloud\" \/>\n<meta property=\"article:published_time\" content=\"2024-03-24T12:22:06+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-04-05T00:31:23+00:00\" \/>\n<meta name=\"author\" content=\"\u967d, \u5411\u5b87\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u57f7\u7b46\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"\u967d, \u5411\u5b87\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/\",\"url\":\"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/\",\"name\":\"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9 - Blog - Silicon Cloud\",\"isPartOf\":{\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/#website\"},\"datePublished\":\"2024-03-24T12:22:06+00:00\",\"dateModified\":\"2024-04-05T00:31:23+00:00\",\"author\":{\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/#\/schema\/person\/4aef6196128551a0f5d66db46c42866c\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/#breadcrumb\"},\"inLanguage\":\"ja\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\/\/www.silicloud.com\/ja\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/#website\",\"url\":\"https:\/\/www.silicloud.com\/ja\/blog\/\",\"name\":\"Blog - Silicon Cloud\",\"description\":\"\",\"inLanguage\":\"ja\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/#\/schema\/person\/4aef6196128551a0f5d66db46c42866c\",\"name\":\"\u967d, \u5411\u5b87\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/ad96b120e219751fa368cbd7d2c48ca01b59185e6ed394449bac72614a760bf3?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/ad96b120e219751fa368cbd7d2c48ca01b59185e6ed394449bac72614a760bf3?s=96&d=mm&r=g\",\"caption\":\"\u967d, \u5411\u5b87\"},\"url\":\"https:\/\/www.silicloud.com\/ja\/blog\/author\/hinatasora\/\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"ja\",\"@id\":\"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/#local-main-organization-logo\",\"url\":\"\",\"contentUrl\":\"\",\"caption\":\"Blog - Silicon Cloud\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9 - Blog - Silicon Cloud","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.silicloud.com\/ja\/blog\/java\u306etokenstream\u306e\u4f7f\u3044\u65b9\/","og_locale":"ja_JP","og_type":"article","og_title":"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9","og_description":"Java\u306eTokenStream\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u3092\u89e3\u6790\u3059\u308b\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u5165\u529b\u30c6\u30ad\u30b9\u30c8\u3092\u30c8\u30fc\u30af\u30f3\u306b\u5206\u89e3\u3057\u307e\u3059\u3002\u5404\u30c8\u30fc\u30af\u30f3\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u5185\u306e\u5358\u8a9e\u307e\u305f\u306f\u8a18\u53f7\u3092\u8868\u3057\u307e\u3059\u3002TokenStream\u3092\u4f7f\u7528\u3059\u308b\u305f\u3081\u306e\u624b\u9806\u306f\u6b21\u306e\u3068\u304a\u308a\u3067\u3059\u3002 [&hellip;]","og_url":"https:\/\/www.silicloud.com\/ja\/blog\/java\u306etokenstream\u306e\u4f7f\u3044\u65b9\/","og_site_name":"Blog - Silicon Cloud","article_published_time":"2024-03-24T12:22:06+00:00","article_modified_time":"2024-04-05T00:31:23+00:00","author":"\u967d, \u5411\u5b87","twitter_card":"summary_large_image","twitter_misc":{"\u57f7\u7b46\u8005":"\u967d, \u5411\u5b87"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/","url":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/","name":"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9 - Blog - Silicon Cloud","isPartOf":{"@id":"https:\/\/www.silicloud.com\/ja\/blog\/#website"},"datePublished":"2024-03-24T12:22:06+00:00","dateModified":"2024-04-05T00:31:23+00:00","author":{"@id":"https:\/\/www.silicloud.com\/ja\/blog\/#\/schema\/person\/4aef6196128551a0f5d66db46c42866c"},"breadcrumb":{"@id":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/#breadcrumb"},"inLanguage":"ja","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.silicloud.com\/ja\/blog\/"},{"@type":"ListItem","position":2,"name":"Java\u306eTokenStream\u306e\u4f7f\u3044\u65b9"}]},{"@type":"WebSite","@id":"https:\/\/www.silicloud.com\/ja\/blog\/#website","url":"https:\/\/www.silicloud.com\/ja\/blog\/","name":"Blog - Silicon Cloud","description":"","inLanguage":"ja"},{"@type":"Person","@id":"https:\/\/www.silicloud.com\/ja\/blog\/#\/schema\/person\/4aef6196128551a0f5d66db46c42866c","name":"\u967d, \u5411\u5b87","image":{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.silicloud.com\/ja\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/ad96b120e219751fa368cbd7d2c48ca01b59185e6ed394449bac72614a760bf3?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ad96b120e219751fa368cbd7d2c48ca01b59185e6ed394449bac72614a760bf3?s=96&d=mm&r=g","caption":"\u967d, \u5411\u5b87"},"url":"https:\/\/www.silicloud.com\/ja\/blog\/author\/hinatasora\/"},{"@type":"ImageObject","inLanguage":"ja","@id":"https:\/\/www.silicloud.com\/ja\/blog\/java%e3%81%aetokenstream%e3%81%ae%e4%bd%bf%e3%81%84%e6%96%b9\/#local-main-organization-logo","url":"","contentUrl":"","caption":"Blog - Silicon Cloud"}]}},"_links":{"self":[{"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/posts\/126117","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/comments?post=126117"}],"version-history":[{"count":2,"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/posts\/126117\/revisions"}],"predecessor-version":[{"id":319012,"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/posts\/126117\/revisions\/319012"}],"wp:attachment":[{"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/media?parent=126117"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/categories?post=126117"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.silicloud.com\/ja\/blog\/wp-json\/wp\/v2\/tags?post=126117"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}