Mirror/dhtcrawler2
1
0
Fork 0
mirror of https://github.com/btdig/dhtcrawler2.git synced 2024-12-13 03:56:21 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
39 Commits 2 Branches 0 Tags 3.4 MiB
269584c708
Commit Graph

39 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kevin Lynx
269584c708 add rmmseg, to segment chinese texts, add a tool to convert the existing torrent file names 2013-07-13 11:45:55 +08:00
Kevin Lynx
676d354515 disable numid for sphinx default 2013-07-12 10:27:23 +08:00
Kevin Lynx
6ddb9447ac Merge branch 'master' of github.com:kevinlynx/dhtcrawler2 
Conflicts:
	ebin/dhtcrawler.app
	ebin/tor_download.beam
 2013-07-12 09:22:09 +08:00
Kevin Lynx
f5965304f7 add torrent download stats for hash reader 2013-07-11 22:38:39 +08:00
Kevin Lynx
1320002674 integrate torrent downloader monitor, change http today_top to show the today request count, instead total request count, remove ibrowse initial config 2013-07-11 22:01:47 +08:00
Kevin Lynx
5a0b21c7b0 chang http top query, add a new database to map date to hashes, to support query by date range 2013-07-11 20:35:16 +08:00
Kevin Lynx
cda02229ad add tor_download req monitor, not integrated yet 2013-07-11 17:50:32 +08:00
Kevin Lynx
42b32810c6 torbuilder(importer) fix badarith bug when there're invalid name torrent 
files
 2013-07-11 09:06:58 +08:00
Kevin Lynx
4adc0a7df4 change http start to pass dbpool size 2013-07-09 22:38:34 +08:00
Kevin Lynx
9982141e7b change hash_reader shell startup script 2013-07-09 21:50:12 +08:00
Kevin Lynx
164c0f0f21 change inc_announce response fileds to empty (only _id) 2013-07-09 21:43:26 +08:00
Kevin Lynx
aa7e8bb18a use safe insert in torrent importer 2013-07-09 16:58:50 +08:00
Kevin Lynx
40dbbeb581 fix hash_reader stop working bug when there's only wait_download hash 2013-07-09 15:06:13 +08:00
Kevin Lynx
03f98c35be update torrent importer, save process state so that can launch next time 2013-07-08 22:16:26 +08:00
Kevin Lynx
0c6097f130 separate embedded css style to page.temp 2013-07-08 12:43:27 +08:00
Kevin Lynx
00cfa07d88 add a tool to add local torrents to database 2013-07-07 19:52:20 +08:00
Kevin Lynx
b42a85e929 fix hash_reader stop bug (send a wrong message) 2013-07-06 21:24:44 +08:00
Kevin Lynx
d17d78ec08 add config to control whether load torrents from database cache 2013-07-06 09:41:51 +08:00
Kevin Lynx
cedd378fa6 IMPORTANT: add number id to torrent database 2013-07-06 01:19:06 +08:00
Kevin Lynx
01f7fd1aaf stats nice print 2013-07-05 23:49:35 +08:00
Kevin Lynx
48022521c9 change torrent path config name 2013-07-05 23:08:26 +08:00
Kevin Lynx
ad8c931c80 stats on torrent cache used 2013-07-05 22:52:15 +08:00
Kevin Lynx
dc8cab211f fix torrent download memory leak bug 2013-07-05 21:53:22 +08:00
Kevin Lynx
7c777f64b6 add torrent downloader 2013-07-05 21:07:35 +08:00
Kevin Lynx
b598fb02ea add torrent cache, not tested now 2013-07-03 22:55:17 +08:00
Kevin Lynx
7b811ddd32 add http short keyword checking 2013-07-03 22:35:12 +08:00
Kevin Lynx
0fcf9f8dda make the cache updating process only one, and make stats to cache 2013-07-03 21:58:06 +08:00
Kevin Lynx
10c11df943 fix http cache bug 2013-07-03 20:02:28 +08:00
Kevin Lynx
d82839ad27 async update cache 2013-07-03 17:53:42 +08:00
Kevin Lynx
4ca1885320 update http cache in a process 2013-07-03 17:46:15 +08:00
Kevin Lynx
c1f99eccd5 add http search cache 2013-07-03 17:19:28 +08:00
Kevin Lynx
494c658aec readme 2013-07-02 20:38:53 +08:00
Kevin Lynx
603336cfd7 update readme 2013-07-02 20:37:14 +08:00
Kevin Lynx
ea6df3b2df add database repl-set scripts 2013-07-02 17:42:35 +08:00
Kevin Lynx
2d4d3f89a7 change http to read data from mongodb slave 2013-07-02 13:57:44 +08:00
Kevin Lynx
c482202fb0 change hash reader wait time to 1 min 2013-07-02 12:27:19 +08:00
Kevin Lynx
9e38df1972 add kdht ebin 2013-07-02 11:14:42 +08:00
Kevin Lynx
2788de582d change http interface 2013-07-02 11:05:07 +08:00
Kevin Lynx
4f7fe8805e first commit 2013-07-01 22:42:14 +08:00