forked from imapsync/imapsync
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathTODO
1045 lines (755 loc) · 35.3 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
#!/bin/cat
# $Id: TODO,v 1.215 2019/06/20 10:12:00 gilles Exp gilles $
This documentation is also at http://imapsync.lamiral.info/#doc
===========================================================================
TODO file for imapsync
----------------------
SUGGESTED 2019_06_18 by Konrad Wawryn
Add NTLMv2 support
It shouldn't be that hard to support NTLMv2 because
the underlying NTLM module can do it already.
Delegate all the stuff to underlying Mail-IMAPClient.
SUGGESTED 2019_06_05 by Gilles
Code equivalent W/tools/fix_email_for_exchange.py
inside imapsync
SUGGESTED 2019_04_10 by Gilles
With signals show also their numbers
https://perldoc.perl.org/Config.html
SUGGESTED 2019_04_03 by Katja Wolf and Gilles
Put a summary like the one in the log file
in the source and the destination mailbox.
SUGGESTED 2019_03_19 by Gilles
In the final errors listing, add lines
* Host1 banner
* Host2 banner
* Command line parameters
I'm a little fade up to ask them
in order to understand the context
SUGGESTED 2019_03_04 by Gilles
Write a FAQ about deciphering with imapsync.
Test it!
If the encryption is made using gpg
then the decryption via --pipemess could be just the command:
imapsync ... --pipemess 'gpg --decrypt --no-batch --no-tty --override-session-key 9:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
where you previously ran:
gpg --decrypt --no-batch --show-session-key some_rfc822_filecrypted.txt
in order to get the session-key "9:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
SUGGESTED 2019_02_28 by Gilles
Default tmpdir could become ./TMP_imapsync
for --usecache if there is no cache in the
historical tmpdir File::Spec->tmpdir( )
Why? Some systems like Windows do change
their tmpdir upon reboot.
Some Unixes do remove their /tmp upon reboot.
SUGGESTED 2018_09_11 by Gilles
Use Smart::Comments in long loop
Use Smart::Comments in debug/verbose mode
SUGGESTED 2018_09_11 by Gilles
Take a look at
Test::NoWarnings
Test::Warn
SUGGESTED 2018_05_15 by Massimo Maggi
https://github.com/imapsync/imapsync/issues/141
Document --skipcrossduplicates and --debugcrossduplicates in imapsync --help
Document all other useful missing options.
SUGGESTED 2018_05_08 by James B. Byrne
Sync also the quota with SETQUOTA
https://tools.ietf.org/html/rfc2087
SUGGESTED 2017_12_19 by Gilles
In foldersizes, add info about the oldest message found: INTERNALDATE EPOCH DAYS_FROM_NOW
See draft in sub info_date_from_uid
SUGGESTED 2017_09_04 by Gilles
STDOUT instead of STDERR with Mail::IMAPClient output.
Shoul be easy using $imap->Debug_fh($fileHandle);
SUGGESTED 2017_08_31 by Gilles
Makefile. Add a "make docker" goal that build, test and publish the
docker package on ks3.
SUGGESTED 2017_08_22 by Ismael Baena
when hmailserver hierarchy delimiter is “\”
When imapsync asks hmailserver about namespace, hmailserver returns “\\” instead of “\”.
After that, imapsync fails when accessing any folder. Comparing hmailserver logs between
imapsync and outlook accessing the account, I see outlook quoting every path
while imapsync don’t, and outlook doesn’t fail.
I wrote a message to the hmailserver forum,
https://www.hmailserver.com/forum/viewtopic.php?f=7&t=31503
SUGGESTED 2017_05_04 by Gilles
Verif ipv6 connection, seems to fail now
./imapsync --host1 2603:1026:200:52::2 --host2 2603:1026:200:52::2 --justconnect
...
Host1: connecting on host1 [2603:1026:200:52::2] port [143]
Host1: Can not open imap connection on [2603:1026:200:52::2]: Unable to connect to 2603:1026:200:52::2: Invalid argument
SUGGESTED 2017_05_04 via github
Review the "make install" in Makefile
SUGGESTED 2017_04_26 by Franco Fassio
--usecache with trailing dots foo... in folder name breaks on NTFS.
SUGGESTED 2017_04_26 by Franco Fassio
Check that --maxbyteperseconds does not interfer with --timeout
by sleeping too long.
DONE by Gilles revision 1.795 date: 2017/04/22
SUGGESTED 2016_12_01 by Gilles
When working under with webmin and virtualmin imapsync
acts as a cgi, it should not. Find a way to avoid cgi mode
even under a web server. Solution: edit the sub under_cgi_context()
SUGGESTED 2016_11_16 by Flávio Zarur Lucarelli LucaNet Sistemas
Getting subjects of messages for duplicates and print them.
Dates and sizes are easier printable because imapsync gets them each time.
SUGGESTED 2016_10_24 by Gilles
Verify regex --delete2foldersbutnot and --delete2foldersonly are ok
before anything.
SUGGESTED 2016_10_07 by Gilles
Write a FAQ file for packagers,
mostly how to get the dependency Perl modules complete.
SUGGESTED 2016_10_07 by Gilles
Write a FAQ file about all errors and explain what they mean.
Maybe propose an action to solve them in the code,
or even solve them directly in the execution.
SUGGESTED 2016_10_06 by Gilles
Document or fix behavior (do a AND),
curently "--search" makes "--maxage" and "--minage" ignored.
SUGGESTED 2016_09_16 by Gilles
Add a check of all --regextrans2 before going further,
like what is done with --regexmess
SUGGESTED 2016_08_25 by David Carter
It would be helpful if imapsync reported Message-Id:/From:/Subject
as well as folder name and message UID when it skips a message.
SUGGESTED 2016_09_02 by Gilles & David Carter
Exchange Online errors that need a relogin:
"BAD User is authenticated but not connected."
"error while reading data from server: Connection reset by peer"
"timeout waiting 1200s for data from server"
SUGGESTED 2016_08_31 by Gilles
Rename $expungeaftereach $expungeaftereach1 or $expunge1aftereach
SUGGESTED 2016_08_31 by Gilles & David Carter
Detect equivalent folders by upper/lower case on host1
and add a way to not merge them when host2 is case insensitive.
SUGGESTED 2016_08_19 by Gilles
Go back to SSL_VERIFY_PEER but include
SSL_ca_file inside imapsync or near.
SUGGESTED 2016_08_07 by Gilles
Add a meaningful exit value to all die:
* fatal software dependency, Perl modules
* fatal parameter issue
* fatal connect issue
* fatal login issue
* fatal permission file issue (open)
* fatal IMAP issue
* fatal IMAP disconnection
Maybe replace all die by exit
Add a meaningful exit value to all exit
* exit at end but with errors
* exit at middle because of errormax
* exit by signal
SUGGESTED 2016_07_29 by Gilles from CSS3 Coursera course
Take a look at
https://modernizr.com/
SUGGESTED 2016_07_29 by Gilles
Move website to HTML5
https://about.validator.nu/
SUGGESTED 2016_07_12 by Fronik
With --automap apply the mapping to subfolders of mapped folders.
If
Sent => Envoyes
then
Sent.Foo => Envoyes.Foo
Sent.Foo.Bar => Envoyes.Foo.bar
SUGGESTED 2016_07_07 by Jean-Dominique Delyon.
Add a way to know easily which account transfers went wrong.
Générer un fichier des comptes qui ont rencontré des problèmes
et afficher le contenu à la fin de la boucle sur les comptes.
SUGGESTED 2016_06_29 by Gilles
Clarify system flags in RFC
Example to add:
If \Forwarded is not in PERMANENTFLAGS but \* is, then transform
\Forwarded to Forwarded. And by default.
SUGGESTED 2016_06_13 by David Carter
--pipemess could also treat stderr
` $command < $input_tmpfile > $output_tmpfile 2> $error_tmpfile `
SUGGESTED 2016_06_22 by Gilles
Add --quiet mode
Make --quiet mode being switched by a signal
SUGGESTED 2016_06_12 by Gilles
Fix many perlcritic, all?
DONE * 60 perlcritic Double-sigil dereference (Severity 2)
DONE * 627 perlcritic violations of CodeLayout::ProhibitTrailingWhitespace. (Severity 1).
DONE * 458 perlcritic violations of ValuesAndExpressions::ProhibitInterpolationOfLiterals. "Useless interpolation of literal string"
DONE * 420 perlcritic violations of CodeLayout::ProhibitParensWithBuiltins. "Builtin function called with parentheses". (Severity: 1).
SUGGESTED 2016_06_09 by Gilles and David Carter
In order to avoid useless headaches from --regexmess, add
--regexheader
--regexbody
SUGGESTED 2016_06_09 by David Carter
Add --pipemess-on-fail
If, and only if, the target server responses with NO or BAD then have another go
with a sanitized version of the message.
SUGGESTED 2016_06_01 by M. Beaubien
Add flag --noheaderwarnings or better name
no header by parse_headers so taking whole header with BODY.PEEK[HEADER]
no header found so adding our own [Message-Id: <151648@imapsync>]
SUGGESTED 2016_04_17 by Gilles
Add a --passfile to allow user=>password style file.
WANTED 2016_03_11
Add a FAQ about Authentication failures and quoting.
WANTED 2016_02_10
Add stats about "Messages found crossduplicate on host1"
WANTED 2016_01_28 Stephen Sookdeo
Able to send email without attachment (in case size is the problem) and maybe note in main body that attachment was removed...
See http://unix.stackexchange.com/questions/174707/remove-delete-attachments-from-email-server-imap
http://unix.stackexchange.com/a/174726/89270
imapsync ... --pipemess 'remove_attachments --sizemin 10MB --addnote "Attachment was removed" '
WANTED 2016_03_06
Add a real test to authenticate with a ssl certificate.
WANTED 2016_03_08
Check if --ssl1 and --tls1 are used together and warn abort and abort in that case.
Same for --ssl2 and --tls2
Start a wiki for imapsync.
Write a Windows tutorial, TUTORIAL_Windows.html.
Write a good practices migration tips document, GOOD_PRACTICES.html
Add uidexpunge with --delete if possible (like with --delete2).
Add the possibility to specify a unity with --maxsize and --minsize
Example --maxsize 10MB --minsize 10KB
B
KB
MB
GB
TB
WANTED 2015_12_16 Gilles Lamiral
Add "df -i" with usecache and abort if the number of messages
to transfer will exhaust empty inodes used by the cache.
Looks like module Filesys::DfPortable will help for Unix and Win32.
==== Similar skip/error messages ====
WANTED 2016_01_28 Stephen Sookdeo
List all emails that gives errors so you know exactly which. List with detailed info.
Then ability to perform action on these specific emails like delete or ignore per email.
WANTED 2015_06_02 Karen F Bath.
Add skipped messages in the final dump.
Print the list of messages not copied and why (duplicates or void header).
I would like to request if you could add additional errors to the bottom,
as we find that things like MaxLineLength and maxsize limit are classed
as skipped messages and in my opinion are errors; as the email message
is not transferred but this is not logged at the bottom.
We have our own scan script which we run on all log files at the end
and copy the users logs into subfolders that have issues.
I've attached a list of things we search for.
"Error", "Output"
NO Mailbox already exists, "Folder"
NO LOGIN failed, "NoLogin"
Could not create folder, "Linux"
Error sending, "Linux"
Write failed 'no error caught', "Linux"
line length exceeds maxlinelength, "Linux"
BAD Command Argument Error. 11, "Linux"
Failure: can not open, "Linux"
No not connected, "Linux"
Write failed 'Connection reset by peer', "Linux"
could not be fetched:, "Linux"
NO Mailbox does not exist,"Error"
exceeds maxsize limit 20971520 bytes,"Error"
error while reading data from server: Connection reset by peer (4x),"Error"
Folder [Inbox] already exists,"Error"
WANTED Add a well described problem for each problem detected
and counted in error counter statistics.
==== End Similar skip/error messages ====
WANTED 2015_05_18
I'd like to be able to print the messages subjects, on some logs:
For instance:
msg INBOX.Archived/95551 marked \Deleted on host2 [GyDD6WpsFEtyBzNFnv]
msg [Gmail]/All Mail/281079 copying to INBOX.Archived
It would be great to be able to print the email subject, in order to debug or to get more useful information.
WANTED 2015_04_25
Add duplicates test option.
WANTED 2015_03_06
Dealing with Content-Type Message/Partial
Extract the components of the partial messages and construct them
with reformime as one message which can then be transferred. (Larry Moore)
See also uudeview http://www.fpx.de/fp/Software/UUDeview/
(Larry said uudeview is weird on Partial issue, too old maybe)
Apply --disarmreadreceipts only to UNSEEN messages.
Add an exit value when exiting because of --exitwhenover
The goal is to know easily why to restart later.
Write a Mail::imapsync package and use it.
One day, when I have really nothing better to do, evaluate:
http://www.rackspace.com/apps/email_hosting/migrations
http://www.yippiemove.com/
http://www.migrationwiz.com/
http://www.microsoft.com/download/en/details.aspx?id=1329 "Microsoft Transporter Suite"
Add --mark-as-deleted1 --mark-as-deleted2 as
aliases for --noexpunge1 --delete1 and --noexpunge2 --delete2
Fix bug found by Pavel Stano on 01/06/2012 (june) imapsync never stop login
when login fails with a "* BYE Temp error" from server.
Consider /var/tmp/ instead of /tmp (/tmp is destoyed
on some Unix at reboot)
Fix long path over than 260 character on Win32 with --usecache
Fix inode crunching with --usecache
Think about Digest::SHA or Digest::SHA::PurePerl.
Think about a file database like DBM instead.
Look at https://en.wikipedia.org/wiki/Radix_tree (David M. advice)
Look at leveldb (Simon Th actux advice)
Look at http://fallabs.com/kyotocabinet/ (JYB advice)
Also Redis (Ludovic Danigo advice)
I am writing now to suggest that imapsync provide info containing the parameters
with which a message is determined to be a duplicate, so that if I wanted to confirm
they are in fact duplicates by looking at the messages themselves, enough info would
be available to easily locate them. One or two duplicates per folder are easy to
find, but I had one large folder with quite a few, and in this case it would have
been a big help.
Explain that users can win time/bandwidth by using --expunge
Fix "\Forwarded" flag bug in courier.
Does \lalala can be forbidden (courier does a
"16 NO Error in IMAP command received by server"
with
* OK [PERMANENTFLAGS (\* \Draft \Answered \Flagged \Deleted \Seen)] Limited
Add sync imap keywords. Sync Gmail labels to imap keyword
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=503159
http://www.linux-france.org/prj/imapsync_list/msg00022.html
http://mail.google.com/support/bin/answer.py?hl=en&answer=77657
http://mail.google.com/support/bin/answer.py?answer=78892
Add and option to sync to & from files.
Add an --aclregextrans2 flag.
"Today we discovered, that Cyrus and Dovecot use different characters for
their ACLs. Syncing ACLs vom Cyrus to Dovecot (at least 1.2) doesn't
work. Cyrus uses c and d, Dovecot uses k and x instead."
Peer Heinlein.
Add more information about skipped messages.
Add Rick Romero patch with
--quiet No output at all
--showstats intended for use with --quiet
Take a look at Mail::IMAPTalk
Simon Bertrang said "way better performance, less problems, easier to use and no
issues so far". Sounds good!
imapsync doesn't report well. It should says "I had
to sync 123 messages but I could transfer only 99 messages"
Maybe count messages not transferred because they're dupplicate.
Fix bug "not possible to use space in the imap password"
Add kerberos authentification
Fix bug with folder names starting with an asterisk: *Archiv
Sending: 220 SELECT *Archiv/Support/foobar
Read: 220 BAD Protokollfehler: "Invalid folder specified in SELECT command"
Add LITERAL+ [RFC 2088] support to imapsync.
Make a m4 file for lsm.imapsync and generate it at each public release.
Add --verbose from Kjetil jumbo patch.
Read the IMAP RFC http://www.faqs.org/rfcs/rfc3501.html
Read:
http://www.washington.edu/imap/documentation/commndmt.txt.html
Add cyrus link about INBOX. namespace
http://asg.web.cmu.edu/cyrus/download/imapd/altnamespace.html
===========================================================================
Now the TODO done! (or not)
===========================================================================
DONE 2019_04_28 by Gilles
SUGGESTED 2016_10_03 by Gilles
Do like tail -F on logfile when
"another imapsync may be curently running",
instead of quit.
Add --tail --notail and --tail on by default in cgi context.
DONE 2019_04_27 by Gilles
SUGGESTED 2019_04_25 by Gilles
Add length of $cache_dir to
Local cache directory: $cache_dir
DONE 2019_04_08 by Gilles
SUGGESTED 2019_03_06 by Gilles
when exit by signal, exit by sending a signal to itself:
How to be a proper program
https://www.cons.org/cracauer/sigint.html
DONE 2018_11_20 by Gilles
Count the messages that are on host2 but not on host1.
(hum... see DONE 2018_09_01)
DONE 2018_09_01 by Gilles
Count messages on host2 that are not in host1
DONE 2018/04/18 revision 1.875 by Gilles
SUGGESTED 2018_06_29 by Michael Hesse
Remove the warning on Windows
“No such signal: SIGUSR1 at script/imapsync line 11744.”
DONE 2017_11_29 by Gilles
SUGGESTED 2017_04_11 by Stefano Lo Cascio
Add a --resyncflags on by default but allowing --noresyncflags
DONE 2017_10_27 by Gilles in revision 1.841
SUGGESTED 2017_09_02 by Gilles
Check that @ARGV is empty after readding it by GetOptions since
remaining arguments are not used and might signal something wrong
in the command line arguments.
DONE 2017_08_31 by Gilles
SUGGESTED 2016_12_14 by Gilles
Look at Pod::Usage and convert the actual doc to it
DONE 2017_04_24 by Gilles
SUGGESTED 2016_11_29 by Gilles
Fix bug passfile no exist => login
Should be fatal sooner.
DONE 2017/03/13 by gilles in revision 1.782
SUGGESTED 2016_06_29 by Gilles
Add --gmail1 --gmail2 --gmail12 to set automatically advices from
https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt
Add --exchange --office365 to set automatically advices from
https://imapsync.lamiral.info/FAQ.d/FAQ.Exchange.txt
Add --domino1 --domino2
DONE by Gilles revision 1.755 date: 2017/01/12
SUGGESTED by Gilles
Try to act as --tls if CAPABILITY allows it and if there is no --notls.
Fallback without --tls if --tls fails.
DONE 2016_09_29 by Gilles
SUGGESTED 2016_04_13 by Gilles
Split --noabletosearch in --noabletosearch1 --noabletosearch2
SUGGESTED 2016_09_29 by Gilles
DONE with revision 1.764 2017/01/19
Try to use TLS or SSL by default unless not asked to do so.
DONE 2016_07_27 by Gilles
SUGGESTED 2016_06_15 by Gilles
Fix sub check_lib_version and its call.
It was ok because of the Perl line "use Mail::IMAPClient 3.30".
SUGGESTED 2016_06_22 by Gilles
DONE 2016_07_29 by Gilles
Make reconnections launched by a signal Ctrl-c, aka INT signal.
2 consecutive Ctrl-c within 1 second then exit the program.
Inline help when Ctrl-c is hit
SUGGESTED 2016_07_15 by Alan Williams
DONE 2016_07_20 for IO::Socket::SSL
Add --inet4 --inet6 to force AF_INET or AF_INET6 ip connexion,
aka ipv4 or ipv6.
SUGGESTED 2016_06_24 by Jean-Dominique Delyon
DONE 2016_06_30 by Gilles
Add --delete1emptyfolders
Destroy host1 empty folders.
Be carefull of non-empty subfolders!
SUGGESTED 2016_07_01 by Gilles
DONE 2016_07_01 by Gilles
Add exit explanation after errorsdump
DONE 2016_01_06. Write a Unix tutorial, TUTORIAL_Unix.html.
DONE by Gilles revision 1.677 date: 2016/01/19
Move --help documentation into the man page so that description is easier to find.
DONE 2015_12_26. WANTED 2015_03_24
Add --sslargs with usage like:
imapsync ... --sslargs 'SSL_version=SSLv3' --sslargs 'SSL_use_cert=1' \
--sslargs 'SSL_verify_mode=SSL_VERIFY_PEER'
See perldoc IO::Socket::SSL for all possibilities.
DONE 2015_12_25 Gilles Lamiral
Add --ssldebug 0-4
DONE 2015_10_06. Find a way to avoid passwords in --debugimap unless needed.
Proposed in Mail-IMAPClient via $imap->Showcredentials()
in patch at
See https://rt.cpan.org/Public/Bug/Display.html?id=107592
DONE 2015_12_03 WANTED 2015_11_24 Jens Herrmann
Add --logdir to allow imapsync choosing the filename while allowing
user to choose the dirname part.
DONE. 2015_08_10.
Guess separators and prefixes when NAMESPACE is not available,
instead of forcing the user to guess and set them.
DONE 2015_08_03. WANTED 2015_07_21
Fix W/learn/imap_utf7_encode
--regextrans2 "s,El&AOk-ments,&AMk-l&AOk-ments,"
DONE. Add a FAQ entry about long path over than 260 character on Win32.
DONE. Build and distribute a standalone Darwin Mac OS X binary.
It's called imapsync_bin_Darwin
DONE. Add a NOP for host2 for each fake copy in --dry mode.
Goal is to avoid timeouts happening only because of --dry
DONE 2015_05_09 WANTED 2015_04_25
Add an option "--subfolder2 FOO" to move all folders to a subfolder FOO:
On Windows:
--regextrans2 "s,${h2_prefix}(.*),${h2_prefix}FOO${h2_sep}$1," \
--regextrans2 "s,^INBOX$,${h2_prefix}FOO${h2_sep}INBOX,"
DONE. Quota are read if available and warning is printed if 90% quota reached.
Can you setup an option to make it stop if the destination mailbox reports
that it is over quota?
DONE. Add stdin/stdout filter before transfer:
"Now i asked me, how to modify your perl program to work with
that - in example, to write each mail to stdout, pipe that to the
conversion program, and read the result from stdin - and this all before
the mail will transfer to the target imap-server"
http://www.courier-mta.org/maildrop/reformime.html
Look at IPC::Open2, assuming a good and safe pipe,
or write more code handling all the bad stranger behaviours possible.
Option name: --pipemess "command arg1 arg2 ..."
DONE. Transform messages with too long linelength.
Office365 and Exchange abort connexion
when they encounter a message with more than 10000 characters on a single line.
Could be done with the previous --pipemess with reformime.
Thanks to Larry Moore for suggesting reformime tool!
Usage: --maxlinelength 9900 maxlinelengthcmd 'reformime -r7'
DONE. Add DavMail in Similar Softwares section. http://davmail.sourceforge.net/
DONE. Convert folder names to utf-8 and print them next to the utf-7 ones.
Look at imapsync/W/learn/
./imap_utf7 data_utf7
DONE. Separate the online site in a folder site/ or www/ and make the upper directory simple.
DONE. --skipemptyfolders
Is it possible to run imapsync in a way that empty folders on host1
aren't created at all on host2? Yes with --skipemptyfolders
NO. Add an option to remove empty folders on host1, except INBOX.
Use --skipemptyfolders instead
DONE. Subject is output when APPEND fails. Done since a long time.
Suggestion: it's very difficult to track down messages which are behaving
funny during the sync. It would be great - and presumably easy to code -
to have an option to have imapsync display e.g. the subject of an
e-mail when it gets synced, rather than just the message ID and the date/time.
DONE. Add W/learn/imap_utf7 in imapsync, the folders lists output looks a good place,
if transcription is needed.
DONE. Add options --log and --logfile to log both on screen and to a file.
DONE. Document the tee command solution (Unix and Windows) in the FAQ.
NEVER. Add an option to implement the faq entry about copying a contact folder.
(write a dedicated software instead!)
NEVER. Use examine() instead of select() in --dry mode.
Add a method doing the switch automagicaly.
DONE. Write --disarmreadreceipts that does:
--regexmess 's{\A(.*?(?! ^$))^Disposition-Notification-To:(.*?)$}{$1X-Disposition-Notification-To:$2}igxms'
DONE. With --foldersizes make the listing on host2 independent of case
(avoid "not yet created" if the folder exists in another case string)
DONE. Write --maxmessagespersecond and --maxbytespersecond for staying under Microsoft's throttling policies.
DONE. Inform about --addheader when the problem occurs "(no wanted headers so we ignore this message)"
DONE. Print the timeout value.
DONE. Inform --timeout is is seconds.
DONE. Add an entry to http://lsm.qqx.org/lsm/
See lsm.imapsync
DONE. Fix the mailing-list archive bug with From at
the beginning of a line
http://www.linux-france.org/prj/imapsync_list/msg00307.html
DONE. Remove 2.2.9 Mail::IMAPClient support.
DONE. Add OAUTH autentication support
https://developers.google.com/google-apps/gmail/oauth_overview
DONE. Add --delete1 as an alias for --delete
DONE. Add current date at the beginning of the run, useful when imapsync
doesn't finish with statistics.
DONE. Add automatic convertion or detection when
separator inversion produces an invalid character.
From uw to cyrus, for example :
FoldA/FoldB.ext -> FoldA.FoldB/ext
DONE. Not done since useless now (--useuid)
Add a --skipheaderinfolder option
DONE. Not fixed since only reported once a long time ago.
Fix this:
> - Erreur avec la traditionnelle différence entre Windows
> et Linux sur les retour-chariots : le calcul de la
> longueur du message ou des entêtes à envoyer au serveur
> cible n'est pas bon sur une machine Windows.
> Ci-dessous la modif :
>
> # No NL Count on Windows my $length = ( -s $file ) + $bare_nl_count;
> my $length = ( -s $file );
I wonder if it is Windows or the imap server used.
DONE. No Debian package anymore.
Add debian packaging in the Makefile.
Write to the debian maintener about that.
DONE with --addheader
Add an option to solve syncing Sent folder when no good header
are available:
http://www.linux-france.org/prj/imapsync_list/msg01151.html
http://www.linux-france.org/prj/imapsync_list/msg01158.html
DONE. Add a note about
"One other thing: You might want to warn idiots like me, that if your
cache resides on a filesystem with a limited number of inodes such as
ext4, your inodes will be exhausted really fast (check with df -i). I
now use a dedicated partition with reiserfs."
DONE. Add an option --exitwhenover 2500000000 to avoid to be banned or locked
http://support.google.com/a/bin/answer.py?hl=en&answer=1071518
DONE. Check if a message exists before fetching it (could have been deleted recently)
(Check avoided when UID mode is not available)
DONE. Change default --useheader 'Message-ID' --useheader Date to
--useheader 'Message-ID' --useheader 'Received'
because Exchange changes Date header! (but not Received:)
DONE. Add "output to reflect everything that imapsync was doing".
Not everything but flag synchronization will be nice"
--debugflags does this.
DONE. Add a script to install automaticaly all needed Perl modules
if needed. Not a script but command lines in INSTALL file.
DONE. Post on imapsync mailing-list when a new release comes.
No since I no longer distribute it gratis, imapsync detects
itself a new release and buyers are informed by a specific
list. I know some packagers (FreeBSD) use the Changelog file
to detect an update.
DONE. Subscribe users to imapsync_update mailing-list automatically after the payment.
(done in paypal_send)
DONE. NO became porn! Post on newsgroup comp.mail.imap when a new release comes.
http://groups.google.fr/group/comp.mail.imap
DONE. The --search option allows this (and many other things)
Add option --exclude_messages_with_flag
DONE. (yahoo login is standard now)
Add yahoo imap support:
http://elearningcentral.blogspot.com/2010/07/how-to-use-imapsync-with-yahoo-imap.html
http://www.bwebcentral.com/utils/imapsync-yahoo
See patches/imapsync-yahoo
DONE. Add --subscribe by default.
Subscribe only when needed (not already subscribed).
DONE. Ask Nick Czeczulin why he wrote patent "Method for mailbox migration"
http://www.google.com/patents/about?id=qI_IAAAAEBAJ
DONE. Add --search option allowing to select messages with any IMAP SEARCH command.
DONE. Make --usecache works with --maxage --maxsize etc.
DONE. Fix Exchange 2010 SP1 issue with --foldersizes when
host2 folders don't exist. $imap->exists calls STATUS.
Is it RFC compliant or an Exchange bug?
Exchange quit after 10 errors.
DONE. Make --delete2 works with --useuid
DONE. write a comment to http://blog.migrationwiz.com/2010/12/09/imapsync-vs-migrationwiz/
DONE. Read http://bugs.gentoo.org/show_bug.cgi?id=354831
Nice conversation.
DONE. Look https://fedorahosted.org/released/imapsync/
DONE. Take a look at https://bitbucket.org/imapsync/imapsync
DONE. Explain expunge behavior in help message.
DONE. Add --authmd51 --authmd52 to permit authmd5 by host.
DONE. Write option --delete2foldersonly regex.
Example: to permit a sync in a subfolder with --delete2folder
--regextrans2 's#(.*)#NEW/$1#' --delete2foldersonly /^NEW/
DONE. Write option --delete2foldersbutnot regex.
Example: to permit a sync but not deleting folder OLD
--delete2foldersbutnot /^OLD/
DONE. Add cache to speed up transfer. Option --usecache
DONE. There was a time imapsync took the whole message when the
header was bad (empty). But it was bad for speed with big
messages (nowadays there are always big messages in
mailboxes). May be the best is to take a part of the
body. Have to code this.
DONE. Add a --delete2folders option
"When syncing mailboxes with imapsync, is there a way to delete folders in the
target account? The --delete2 option only seems to delete individual
messages, not folders."
DONE. Add NTLM authentification support. Mail-IMAPClient-3.25
support it.
http://cpansearch.perl.org/src/BUZZ/NTLM-1.05/NTLM.pm
http://curl.haxx.se/rfc/ntlm.html
DONE. Evaluate memory consumption with (or better):
print qx{ ps o pid,pcpu,comm,vsz,rss,size $$ }, "\n"
Search memory leaks with
Test-Weaken Test-Memory-Cycle Devel-Cycle Devel-Leak Test-Weaken
sh -x tests.sh ll_bigmail
sh -x tests.sh ll_memory_consumption
are good candidate to stress memory.
No memory leak detected just up to 8 memory copies of the same data
in Mail::IMAPClient.
Wrote ./learn/memory_consumption to show that it is a Mail::IMAPClient issue.
DONE.Be more effiscient with large mailboxes
Write a Mail::IMAPClient::fetch_hash allowing selecting messages to fetch
4 hours estimated time coding. Time spent 4h30 (with public release and emails)
DONE. Bugfix. Duplicate messages on host2 are not deleted with --delete2
Reason: "Skipping msg #120:508 in host2 folder INBOX.2005-INBOX (duplicate so we ignore this message)"
DONE. Ask namespace capability after login (dovecot 2).
It was a bug from library caching CAPABILITY.
DONE. Write a clean_exit() replacing each die() or exit() call.
Wrote exit_clean() and die_clean() to remove pid file.
DONE. Add a --pidfile option.
DONE. Add NOOP commands to avoid timeouts.
Useless since reconnect mode permits timeouts.
DONE. Add --subscribeall option.
Is it possible to have a option that subscribes all folders regardless of
subscription on the source server? Perhaps --subscribeall?
DONE. Add an option to make imapsync automatically
reconnect when the connection drops
DONE. Add Google adsence to imapsync website to see if it can help.
DONE. Be "FLAGS.SILENT" the normal behavior instead of "+FLAGS.SILENT".
DONE. Add TLS support with patches/imapsync-1.217_tls_support.patch
USELESS. Make --skipheader be multiple. Useless since we can
use the or perl regex '^X-|^Date|^From'
USELESS. Add a --tmpfile option.
DONE. Add --justlogin options
to check usernames and passwords.
DONE. Add a --tmpdir option.
DONE. Fix bug "Found that if folders have a space at the end of
the name, it will not create the folder name on the new
server, nor will it copy that folder's email."
Added regression test checking this.
Could not reproduce this bug.
Seems to be a old imapsync bug with 2.2.9
DONE. (contacted Date::Manip maintener) Fix bug with Date::Manip 6.02
DONE. Explain howto change only the header with --regexmess
DONE.
Change IsUnconnected behavior. If IsUnconnected then print
stats and die. Avoid logout.
DONE. Remove --password "secret" in outpout.
DONE. Make "--delete 2" be a fatal error.
DONE.
Write to zimbra coders about the date presentation:
Sorting by default by internal date is bad with migrations
and delayed arrivals. An user option would be good.
DONE.
Add failure return code in case of
last FOLDER if $from->IsUnconnected();
last FOLDER if $to->IsUnconnected();
See Phil Lobbes messages (18 Sep 2008)
Subject: BUG: imapsync may silently disconnect leading to unmigrated data without warnings
DONE. Fix bad english language.
DONE. Add a better test to check non existing folders on destination
server. (status and select are bad).
DONE. Fix the buggy --include behavior with no --folder --folderrec or --subscribed.
DONE. Update the distribution list including imapsync in the
INSTALL chapter. Also in README.
DONE. Start FAQ entries about good options with such imap server.
DONE. Talk about mbox and maildir format when "Invalid header"
problem is encountered. (See FAQ "Invalid header")
DONE. Look at http://barnson.org/node/81
An old blog about imap migration.
DONE. Normally, "no header used or found" means imapsync found a message
with no header. It may be a bad message with really no header
or it may be a imap server problem, the server gives no header
for this message. In that case imapsync gets the whole message
to see if there is the same on the other side. It slows the transfer
of course. I think I'll change imapsync behavior and let it
give up those bad messages missing an header.
DONE. Make --syncinternaldates turn on by default
DONE. Check imapsync with gmail (dates problem?).
DONE. Start an imap internet scan project.
DONE. Fix --ssl --justconnect bug
DONE. Fix --ssl needs --authmech login with gmail. Why?
DONE. Try a 50Mo message transfer (slow ?)
fast!
DONE. Add my amazon wishlist link.
DONE. Remove file rfc2342.txt "IMAP4 Namespace"
DONE. Add an --exactsync option to remove target messages
that are not on source. Maybe --delete2 is a better name.
DONE. Fix the "word too large" or "Command line too long"
bug with some imap servers.
DONE. Add features from Herman (patch against 1.139 and
again with 1.156 by Kjetil Torgrim Homme)
- Support for authenticating as different (admin) user
- Support for different auth mechanisms (we needed PLAIN)
DONE. Make --include --exclude options possibly be a list.
DONE. Look at http://freshmeat.net/projects/freshmeat-submit/
DONE in FAQ.Talk about :
"I think notes is right but MS Eudora is bad with dates.
(netscape or thunderbird is ok with dates)
use the option --syncinternaldates "
DONE. Create a imapsync mailing list.
May be I won't since we can use comp.mail.imap
http://groups.google.fr/group/comp.mail.imap
to talk about imapsync
DONE. Add a --recurse option when --folder option is used.
No I won't since this feature can be done by doing a
--include '^INBOX.MyFolder'
Yes done with --folderrec
Done. Look at comp.mail.imap http://groups.google.fr/groups?q=imapsync
In fact I set up an google alert on imapsync so I read
any posted question about imapsync posted to any group
followed by google.
DONE. Add usage about --regextrans2
To remove INBOX. in the name of destination folders:
--regextrans2 's/^INBOX\.(.+)/$1/'
To sync a complete account in a subfolder called FOO:
--regextrans2 's/^INBOX(.*)/INBOX.FOO$1/'
DONE. Make the --justconnect a real "just connect" connection,
not a auth connection like it is now.
DONE. Add --prefix1 option.
The need is to remove this prefix when building target folder names.
DONE. Read the IMAP RFC http://www.rfc-editor.org/rfc/rfc2342.txt
Respect the NAMESPACE behavior like the RFC.