imap-filter

• imap-filter ** Intoduction imap-filter is a Ruby implementation of an IMAP filtering application. it can handle multiple IMAP accounts, and create IMAP folders automatically where none exists.

  The imap-filter DSL makes it easy to filter. You can also do "dry-runs" to make sure what happens is what is expected.

** Installation This is a Ruby CLI application. If you are running Linux, most likely you have Ruby already installed. If you are running MacOSX or Windows, you may have to Install Ruby. We support Ruby 2.2 and later.

Simply, do:

#+begin_src bash gem install imap-filter #+end_src

or if in a Gemfile:

#+begin_src ruby gem 'imap-filter' #+end_src

By default,

** Documentation I think that to make this documenation truly "complete", I would wind up writing more lines than in the imap-filter code itself!!! I will attempt to be as concise as possible, and besides me writing a thick manual, no one wants to read such either.

*** Tutorial and Examples Let's go though the steps with just one Email account, then we'll add a second account. Then you can look at the sample [[examples/default.imap][default.imap]] file, and go beyond.

This tutorial assumes you have both a Gmail and a Yahoo account. Please feel free
to substitute with your own accounts.

First, install imap-filter as described in [[#installation][Installation]].

Next, create a default.imap file with Emacs, vi, nano, or whatever suits your fancy. Place
in it the following:

#+begin_src ruby
account :gmail do
  login '[e-mail]', 'yourpassword'
  serv "imap.gmail.com"
  auth :plain
end
#+end_src

And obviously adjust your userid and password to your own. Now let's try to run
your new script:

#+begin_src basn
imap-filter filter default.imap -t
#+end_src

Note that the '-t' is the test flag. It will simply test the connections
to your accounts and nothing more.

And nothing happens. Why not? Very simple. The directive of this file
must be activated. Edit your default.imap and add "activate :all" to the very
end:

#+begin_src ruby
account :gmail do
  login '[e-mail]', 'yourpassword'
  serv "imap.gmail.com"
  auth :plain
end

activate :all
#+end_src

Now run the command again:
#+begin_src basn
imap-filter filter default.imap -t
#+end_src

Ah, something happens now! If all goes well, you should see the following:

#+begin_src bash
====== Test Accounts
  gmail...SUCCESS, delim /
#+end_src

The "delim" is the mailbox delimiter for your paricular vendor. Most places
use the forward slash (/). Some might use a period (.). This lets you know what
to use, so make particular note of that.

If something is wrong with your settings, you might see an error message similar to:

#+begin_src bash
====== Test Accounts
  gmail...FAILED: Lookup failed xxxxxxxxxxxxx
#+end_src

And so check your login line to make sure your userID and password is correct.
For some vendors, the userID is your complete email address. For others, it might
just be what procedes the '@'. Please check with your vendor to see what it is for
you. Usually, your vendor will have IMAP Client instructions on their website.

We use ssl connections by default, and it would be extremely insecure not to. But
you can turn off the ssl if need be. You can also change your port settings
accordingly, if your vendor uses a non-standard port. For example,

#+begin_src ruby
account :gmail do
  login '[e-mail]', 'yourpassword'
  serv "imap.gmail.com"
  auth :plain
  ssl true
  port 993
end

activate :all
#+end_src

You may set ssl to "false" or make the port "2001". You have full
control over the settings. auth could be :login or :plain, so try
:login if :plain doesn't work.

There are more extensive authenication methods that IMAP can use, but
we do not support them at this time. If you have need for that support,
please let me know!!!!!

Now, let's assume you have your settings working. Now what? There are no
filters specified, which is the entire point of this program!!! So let's
add one:

#+begin_src ruby
account :gmail do
  login '[e-mail]', 'yourpassword'
  serv "imap.gmail.com"
  auth :plain
end

filter :github, 'gmail:INBOX' do
  search do
    from 'github.com'
    unseen
  end

  cp 'gmail:test/Github'
end

activate :all
#+end_src

Note that the "actvate :all" is always the last entry. The filter
directive is fairly sophisticated, supporing most of what is possible
with IMAP search. We will document this more throughly at a later date.

In this filter example, we are receiving email notifications from github
which is cluttering up the Inbox. This filter will copy the "unseen", or
unread ones to the "test/Github" mailbox.

Notice in the string, we also specify the target account label. This is
required, and will allow you to specify alternate accounts very easily.

The mailbox will automatically be created if it does not exist yet.

So now, let's run it with the following commands. First, we wish to do a
sanity check, so we will do a dry run:

#+begin_src bash
imap-filter filter default.imap -u -v1
#+end_src

The "-u" is the alias for "--dryrun" The "-v1" is the verbosity switch. While
doing your setups and debugging your scripts, you will want to use "-v1" or even
"-v2" verbosity settings.

If you actually have a match for the search critieria, you may see a list of messages
that matched the search scroll by. In my test case:

#+begin_src bash
...
##Subject: Re: [jekyll/jekyll] jekyll serve can't work (#5216)
##Subject: [jruby/jruby] Fix Socket.tcp connect_timeout option (#4061)
##Subject: Re: [ansible/ansible-modules-core] Add state=empty into file module (#902)
##Subject: [ansible/ansible] Publish modules to Galaxy? (#17027)
##Subject: [ansible/ansible-modules-core] Add support for password aging on Solaris (#4372)
##Subject: Re: [jruby/jruby] Fix Socket.tcp connect_timeout option (#4061)
##Subject: Re: [tmux/tmux] Keeps getting "lost server" in tmux 2.1 and 2.2. Installed using Homebrew (#498)
##Subject: [jekyll/jekyll] Add link to Staticman (#5224)
##Subject: Re: [ansible/ansible] Ansible facts override special variables (#16935)
##Subject: Re: [elixir-lang/elixir] Introduce List.pop_at/3 (#5118)
...
#+end_src

Which are all unread or "unseen" as per our specificaions. Now no modifications
were peformed due to the "-u" dry run switch.

Now that we are feeling a bit more confident, let's run the same
command without the "-u" setting:

#+begin_src bash
imap-filter filter default.imap -v1
#+end_src

And now you will find that the filter actually executed what we wanted. A new
test/Gitub mailbox was created, and all the matching messages are now copied there.
Neat, huh?

Well, that's it for the tutorial. Please have a look at the [[examples/default.imap][example]].

There's a shorthand for the filters that may interest you for the
majority of your cases. For example:

#+begin_src ruby
filter :slashdot, 'gmx:INBOX', from: 'slashdot' do
  mark :seen
  cp 'google:INBOX'
  mv 'gmx:Slashdot'
end
#+end_src

Uses the short-hand of "from: 'slashdot'" when all you need
to do is search a single field, like in this case, the From
field.

Others are obvios: to:, cc:, subject: will all work here.no-expand

*** Environment variables | Variable | Description | |-----------------+-------------------------------------------------------------------| | IMAPF_IMAP_FILE | pathname to the default .imap file. The default is ./default.imap |

*** Accessing your Gmail Google may automatically block this application by default. Please visit the follow page to understand how to allow imap-filter to access your Gmail account.

https://support.google.com/accounts/answer/6010255

*** Automatic blocks might happen while you are testing your Email accounts Depending on the vendor, while you are setting up your scripts, you will be doing many tests. Some vendors like Yahoo might block the app if it sees what it consideres to be "too much" activity. If you see this happen, wait about 30 minutes before trying again. Should work in most cases.

*** DSL #+begin_src Ruby account :redbird do login "[e-mail]", "password" serv "imap.redbird.org" ssl false port 993 end

account :yahoo do
  login "[e-mail]", "password"
  serv "imap.yahoo.com"
end

account :google do
  login "[e-mail]", "password"
  serv "imap.google.com"
end

filter :workmail, redbird.inbox, from: 'sally' do
  cp yahoo.girlfriend
  mv google.inbox
end

filter :maillist, google.inbox, from: 'github' do
  mv google.groups.github
end

#+end_src

*** Reference Documents https://tools.ietf.org/html/rfc3501 https://www.ietf.org/rfc/rfc2822.txt

** Release Notes and Known Issues *** Release Notes | Version | Breif | Description | |---------+--------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 0.1.4 | Dangling command removed | There was a list command that I had not implemented yet, but Thor now complains when it sees that. So the dangling command was removed. | | 0.1.0 | election day bug fixed | This was an issue with moves not deleting the sources, and had to do with state information not existing in the duplicate delegator object. This snafu has been dealt with. No more duplicate delegators | | 0.0.4 | "Too many errors" fixed | The "duplicate mailbox" error was being counted by some providers as being "too much" so now we check first before attempting to create a new mailbox. | | 0.0.3 | Fixed bug with timeout | On some lengthy remove copy/move operations, the source account can timeout. We do a simple NOOP to keep the connection alive. | | 0.0.2 | Initial Release | |

*** Known Issues | Date | Description | |------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 2017-09-25 | No spec coverage. I know, I know... | | 2017-09-25 | There is a problem connecting to Yahoo, and it looks like an OAuth support issue from my end, as they apparenly no longer support the "simple" login anymore. Issue is being worked on on a branch for now. | | 2016-11-08 | US election day bug :p -- messages on move are not being moved, only copied with v0.0.4. Refactoring is broken. Use 0.0.3 for now. Fix is coming soon. | | 2016-11-05 | Aside from being Guy Fawkes Day, the fix to "Too Many Errors" involved caching the list of mail boxes. If something else creates a mailbox while this is running, there is a potential race condition, but the name collision, if any, will only result in a "duplicate mailbox" error. | | 2016-09-25 | Remote copy / moves of HTML-based email does not work cleanly. | | 2016-10-03 | Timeout errors still persist in some cases. | | | Need to capture all errors and provide human-readable responses, unless a certail verbosity level has been set. |

** Contributing to imap-filter

• Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet. |
• Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it. |
• Fork the project. |
• Start a feature/bugfix branch. |
• Commit and push until you are happy with your contribution. |
• Make sure to add tests for it. This is important so I don't break it in a future version unintentionally. |
• Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it. |

** Copyright

Copyright (c) 2016 Fred Mitchell. See LICENSE.txt for further details.

** Scratchpad This section is my personal scratchpad. Should be of no revelance to anyone else. Please ignore what's in this section.

:Login: imap = Net::IMAP.new('imap.gmail.com', ssl: true) imap.authenticate('PLAIN', ENV['GOOGLE_EMAIL'], ENV['GOOGLE_PASS']) imap.select('Shelby') seq = imap.search(['ALL']) :END:

:FETCH: acc.imap.fetch(1..2, 'BODY[TEXT]') -- the entire text, flags as well. acc.imap.fetch(1..2, 'BODY[HEADER.FIELDS (SUBJECT)]') :END:

:Syswrite: IOError: closed stream from 2.3.1/lib/ruby/2.3.0/openssl/buffering.rb:322:in `syswrite' :END: