Given a list of links (as a text file), this module will group similar links/URLs (or find unique links) and write them to separate files.
e.g. URL list --
/level1/uid8
/level1/level2/gid6
/status
/level1/uid10
/level1/level2/gid0
/xyz
/level1
/level1
/level1
/level1/level2
e.g. output (given groupsize of 2)
Then this will list the following in file 1 (URLs which could not be grouped) --
/status
/xyz
This will exist in file 2 --
/level1/uid8
/level1/uid10
Contents of file 3 --
/level1/level2/gid0
/level1/level2/gid6
Contents of file 4 --
/level1/level2
conents of file 5 --
/level1
/level1
/level1
Use LinkGrouper.uniqcounter method as the entry point.
1st arg -- The directory in which files will be made each of which represents a group.
2nd arg -- A file which contains a \n separated list of absolute URLs
3rd arg -- The 'groupsize'. A identified category/class/group of URLs must be of this size to be considered a category/class/group.
LinkGrouper_app.rb has been made which is an app using the module.
First arg -- Will write into a directory (plist) with it's path supplied to argument. This contains files each of which represents a class/category/group of log entries. The file is named with a random UUID.
2nd arg -- A file which contains a \n separated list of absolute URLs
3rd arg -- The 'groupsize'. A identified category/class/group of URLs must be of this size to be considered a category/class/group.