Package gohn implements a client for the Hacker News REST API from firebaseio
Package lingua accurately detects the natural language of written text, be it long or short. Its task is simple: It tells you which language some text is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages. Language detection is often done as part of large machine learning frameworks or natural language processing applications. In cases where you don't need the full-fledged functionality of those systems or don't want to learn the ropes of those, a small flexible library comes in handy. So far, the only other comprehensive open source library in the Go ecosystem for this task is Whatlanggo (https://github.com/abadojack/whatlanggo). Unfortunately, it has two major drawbacks: 1. Detection only works with quite lengthy text fragments. For very short text snippets such as Twitter messages, it does not provide adequate results. 2. The more languages take part in the decision process, the less accurate are the detection results. Lingua aims at eliminating these problems. It nearly does not need any configuration and yields pretty accurate results on both long and short text, even on single words and phrases. It draws on both rule-based and statistical methods but does not use any dictionaries of words. It does not need a connection to any external API or service either. Once the library has been downloaded, it can be used completely offline. Compared to other language detection libraries, Lingua's focus is on quality over quantity, that is, getting detection right for a small set of languages first before adding new ones. Currently, 75 languages are supported. They are listed as variants of type Language. Lingua is able to report accuracy statistics for some bundled test data available for each supported language. The test data for each language is split into three parts: 1. a list of single words with a minimum length of 5 characters 2. a list of word pairs with a minimum length of 10 characters 3. a list of complete grammatical sentences of various lengths Both the language models and the test data have been created from separate documents of the Wortschatz corpora (https://wortschatz.uni-leipzig.de) offered by Leipzig University, Germany. Data crawled from various news websites have been used for training, each corpus comprising one million sentences. For testing, corpora made of arbitrarily chosen websites have been used, each comprising ten thousand sentences. From each test corpus, a random unsorted subset of 1000 single words, 1000 word pairs and 1000 sentences has been extracted, respectively. Given the generated test data, I have compared the detection results of Lingua, and Whatlanggo running over the data of Lingua's supported 75 languages. Additionally, I have added Google's CLD3 (https://github.com/google/cld3/) to the comparison with the help of the gocld3 bindings (https://github.com/jmhodges/gocld3). Languages that are not supported by CLD3 or Whatlanggo are simply ignored during the detection process. Lingua clearly outperforms its contenders. Every language detector uses a probabilistic n-gram (https://en.wikipedia.org/wiki/N-gram) model trained on the character distribution in some training corpus. Most libraries only use n-grams of size 3 (trigrams) which is satisfactory for detecting the language of longer text fragments consisting of multiple sentences. For short phrases or single words, however, trigrams are not enough. The shorter the input text is, the less n-grams are available. The probabilities estimated from such few n-grams are not reliable. This is why Lingua makes use of n-grams of sizes 1 up to 5 which results in much more accurate prediction of the correct language. A second important difference is that Lingua does not only use such a statistical model, but also a rule-based engine. This engine first determines the alphabet of the input text and searches for characters which are unique in one or more languages. If exactly one language can be reliably chosen this way, the statistical model is not necessary anymore. In any case, the rule-based engine filters out languages that do not satisfy the conditions of the input text. Only then, in a second step, the probabilistic n-gram model is taken into consideration. This makes sense because loading less language models means less memory consumption and better runtime performance. In general, it is always a good idea to restrict the set of languages to be considered in the classification process using the respective api methods. If you know beforehand that certain languages are never to occur in an input text, do not let those take part in the classifcation process. The filtering mechanism of the rule-based engine is quite good, however, filtering based on your own knowledge of the input text is always preferable. There might be classification tasks where you know beforehand that your language data is definitely not written in Latin, for instance. The detection accuracy can become better in such cases if you exclude certain languages from the decision process or just explicitly include relevant languages. Knowing about the most likely language is nice but how reliable is the computed likelihood? And how less likely are the other examined languages in comparison to the most likely one? In the example below, a slice of ConfidenceValue is returned containing those languages which the calling instance of LanguageDetector has been built from. The entries are sorted by their confidence value in descending order. Each value is a probability between 0.0 and 1.0. The probabilities of all languages will sum to 1.0. If the language is unambiguously identified by the rule engine, the value 1.0 will always be returned for this language. The other languages will receive a value of 0.0. By default, Lingua uses lazy-loading to load only those language models on demand which are considered relevant by the rule-based filter engine. For web services, for instance, it is rather beneficial to preload all language models into memory to avoid unexpected latency while waiting for the service response. If you want to enable the eager-loading mode, you can do it as seen below. Multiple instances of LanguageDetector share the same language models in memory which are accessed asynchronously by the instances. By default, Lingua returns the most likely language for a given input text. However, there are certain words that are spelled the same in more than one language. The word `prologue`, for instance, is both a valid English and French word. Lingua would output either English or French which might be wrong in the given context. For cases like that, it is possible to specify a minimum relative distance that the logarithmized and summed up probabilities for each possible language have to satisfy. It can be stated as seen below. Be aware that the distance between the language probabilities is dependent on the length of the input text. The longer the input text, the larger the distance between the languages. So if you want to classify very short text phrases, do not set the minimum relative distance too high. Otherwise Unknown will be returned most of the time as in the example below. This is the return value for cases where language detection is not reliably possible.
Package hackernews implements the Hacker News API
Package gohn implements a client for the Hacker News REST API from firebaseio
Package nds is a Go datastore API for Google App Engine that caches datastore calls in memcache in a strongly consistent manner.
Package qml offers graphical QML application support for the Go language. This package is in an alpha stage, and still in heavy development. APIs may change, and things may break. At this time contributors and developers that are interested in tracking the development closely are encouraged to use it. If you'd prefer a more stable release, please hold on a bit and subscribe to the mailing list for news. It's in a pretty good state, so it shall not take too long. See http://github.com/go-qml/qml for details. The qml package enables Go programs to display and manipulate graphical content using Qt's QML framework. QML uses a declarative language to express structure and style, and supports JavaScript for in-place manipulation of the described content. When using the Go qml package, such QML content can also interact with Go values, making use of its exported fields and methods, and even explicitly creating new instances of registered Go types. A simple Go application that integrates with QML may perform the following steps for offering a graphical interface: Some of these topics are covered below, and may also be observed in practice in the following examples: The following logic demonstrates loading a QML file into a window: Any QML object may be manipulated by Go via the Object interface. That interface is implemented both by dynamic QML values obtained from a running engine, and by Go types in the qml package that represent QML values, such as Window, Context, and Engine. For example, the following logic creates a window and prints its width whenever it's made visible: Information about the methods, properties, and signals that are available for QML objects may be obtained in the Qt documentation. As a reference, the "visibleChanged" signal and the "width" property used in the example above are described at: When in doubt about what type is being manipulated, the Object.TypeName method provides the type name of the underlying value. The simplest way of making a Go value available to QML code is setting it as a variable of the engine's root context, as in: This logic would enable the following QML code to successfully run: While registering an individual Go value as described above is a quick way to get started, it is also fairly limited. For more flexibility, a Go type may be registered so that QML code can natively create new instances in an arbitrary position of the structure. This may be achieved via the RegisterType function, as the following example demonstrates: With this logic in place, QML code can create new instances of Person by itself: Independently from the mechanism used to publish a Go value to QML code, its methods and fields are available to QML logic as methods and properties of the respective QML object representing it. As required by QML, though, the Go method and field names are lowercased according to the following scheme when being accesed from QML: While QML code can directly read and write exported fields of Go values, as described above, a Go type can also intercept writes to specific fields by declaring a setter method according to common Go conventions. This is often useful for updating the internal state or the visible content of a Go-defined type. For example: In the example above, whenever QML code attempts to update the Person.Name field via any means (direct assignment, object declarations, etc) the SetName method is invoked with the provided value instead. A setter method may also be used in conjunction with a getter method rather than a real type field. A method is only considered a getter in the presence of the respective setter, and according to common Go conventions it must not have the Get prefix. Inside QML logic, the getter and setter pair is seen as a single object property. Custom types implemented in Go may have displayable content by defining a Paint method such as: A simple example is available at: Resource files (qml code, images, etc) may be packed into the Go qml application binary to simplify its handling and distribution. This is done with the genqrc tool: The following blog post provides more details:
Package gosugamers is Crawlora's GosuGamers API client for 17 endpoints, covering esports matches, results, tournaments, games, news, and articles.