Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6dd33bd
Implemented Thompson Sampling
ogentry Feb 6, 2020
a89536d
Updated config file to include TS and added TS import to Simulation.py
ogentry Feb 6, 2020
415a68e
Update Thompson Sampling to choose the k best articles
ogentry Feb 6, 2020
b02e2e2
Begin implentation of LinPHE (Linear Perturbed History Exploration)
ogentry Feb 10, 2020
eb4c091
Update LinPHE to be able to choose k top articles
ogentry Feb 10, 2020
175d705
Minor changes to LinPHE.py
ogentry Feb 24, 2020
0131d6b
Update the default value of paramaters for LinPHE and ThompsonSampling
ogentry Feb 24, 2020
8ec3e5a
Imperfect implementation of BanditLib with rewards coming from dataset
ogentry Mar 22, 2020
1ccd4bd
Fix error with choosing dataset
ogentry Mar 22, 2020
0b1bba4
Record regret at each batch
ogentry Mar 26, 2020
114360f
Changes to filepaths
ogentry Mar 26, 2020
80b1813
All algorithms work with simulate rewards. Update util_functions dict…
ogentry Mar 27, 2020
ea2e32f
Fix COLin and GOBlin to work with LastFM
ogentry Mar 28, 2020
6c6b24a
Implement MLP based algorithm
ogentry Apr 2, 2020
e89fdda
Add createMLPDict function
ogentry Apr 2, 2020
20fef9e
Creation of Perturbed MLP algorithm
ogentry Apr 16, 2020
0a87156
Implement perturbed MLP
ogentry Apr 16, 2020
e91acac
MLP models work much better
ogentry Apr 30, 2020
2fb8f24
Remove Torch Linear Regression, add Cuda
ogentry May 19, 2020
fb73cc5
Add gradient pertubation
ogentry May 20, 2020
7e470fe
MLP algorithms main params are set through config file
ogentry May 24, 2020
cd52645
Can now model on Yahoo dataset
ogentry Jun 18, 2020
67d78f9
Implement UCBMLP algorithm
ogentry Jul 20, 2020
1bb892f
Fix some issues with RewardManagers, add todo file
ogentry Jul 20, 2020
669f571
YahooRewardManager plots pop up, two config files
ogentry Jul 20, 2020
fed852b
Add documentation
ogentry Jul 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Dataset/Yahoo/Yahoo.top80.cut
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1]
[0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1]
160 changes: 160 additions & 0 deletions Dataset/Yahoo/YahooKMeansModel/10kmeans_model160.dat
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
0.0306616721531 0.00449813843946 0.170505844007 0.79245554436 0.0018788000933
0.387018869168 0.00836122065206 0.455198866759 0.143806875318 0.00561416781044
0.0148717083308 0.00243379287671 0.00991121136685 0.0148977674725 0.957885518378
0.498774335953 0.0393418106297 0.164589526291 0.289649462663 0.00764486580873
0.0347075883468 0.856125496581 0.000435120825535 0.106325522482 0.00240627171756
0.041592004217 0.340758577936 0.00443299709454 0.611034720938 0.00218169849079
0.0619671086193 0.000167496096925 0.913063004064 0.0232192540615 0.00158313523747
0.0359253747115 0.00176371146336 0.422961121572 0.537330156597 0.00201963476493
0.0268761583368 0.0124591548465 0.0261815025022 0.235962455949 0.698520728397
0.138513714252 0.512464294778 0.00706300264957 0.338942105837 0.00301688419817
0.342243412474 0.592921583591 0.00308047290903 0.0586543034538 0.0031002288673
0.244412008083 0.697119279346 0.00178804418176 0.054512465991 0.0021682041515
0.386810178225 0.0155572155547 0.354744914404 0.237058937053 0.00582875444137
0.193705085893 0.0613731253075 0.0688408619728 0.67137949259 0.0047014350613
0.51305623505 0.161768933533 0.0477177508596 0.272813198658 0.00464387936648
0.0636758872799 0.00530365085933 0.219949526363 0.464306647889 0.246764284744
0.798741082472 0.0461258771995 0.0525970008071 0.0916066179891 0.0109294235523
0.312394598309 0.00214246204986 0.635961849958 0.0463263635559 0.00317472757089
0.140317534483 0.608667431658 0.00460441622671 0.242529146873 0.00388146734779
0.117612414913 0.00230352978309 0.670777080854 0.206525213412 0.00278176024928
0.0122147494356 0.976873633426 2.0189660336e-05 0.0105711834103 0.00032019841226
0.0422856860721 0.782497789254 0.000838843292343 0.172429187607 0.00194849489877
0.332349831908 0.0657128872208 0.0761988605401 0.518877956788 0.00686046523232
0.477952535501 0.00483171557369 0.4595558659 0.0529793816031 0.00468050218286
0.114848195544 0.00868981073701 0.330603505218 0.542721392009 0.00313709682607
0.0234378279695 0.000258289625799 0.822391186985 0.152960841783 0.00095185279311
0.134494522237 0.239650765969 0.0207059047995 0.60248911563 0.00265969122987
0.0954572588407 0.300009543225 0.000796140939426 0.0733801125061 0.530356944412
0.0327339851653 0.000124648725607 0.284047303883 0.0283342543011 0.654759800657
0.090436752059 0.0994615413235 0.0297350232063 0.778362720769 0.00200396230624
0.64903584185 0.0100545348447 0.279568870903 0.0549436756161 0.00639707572668
0.0299827422108 0.116334445067 0.00905413987092 0.843007745799 0.0016209265475
0.0346610535337 0.0255943593812 0.0398093618398 0.624182458818 0.275752769072
0.636192729149 0.284821984041 0.0114740421739 0.0647120407127 0.00279920511263
0.810080611782 0.112862055582 0.0217101045461 0.0508709700965 0.00447625969525
0.0888601577404 0.000885527944865 0.80452720635 0.104051790243 0.0016753180522
0.0925888708142 0.0268007218866 0.0134341994629 0.0349375733334 0.832238635417
0.0413688331383 0.612186444146 0.00177349393921 0.342404555793 0.00226667466103
0.123135174528 0.0613911346806 0.0246618074365 0.18425128869 0.606560587937
0.00468886761749 3.62962773595e-06 0.988498661489 0.00649151668963 0.000317283935118
0.0338406081866 0.0111660421862 0.0276111986323 0.13688730105 0.790494850894
0.477096753986 0.0123056517109 0.362175049017 0.142140189772 0.00628235631767
0.205476925883 0.00480829985106 0.577911542234 0.208182256444 0.00362097518876
0.429489750533 0.0424952688076 0.152187778833 0.368963860318 0.00686334356562
0.231540349301 0.373083987893 0.0170910173629 0.375527844523 0.00275680094016
0.0356782015044 0.00226296206844 0.353595556404 0.60645691109 0.00200636788875
0.0503748846981 0.00116366491333 0.438304862343 0.245690140167 0.26446644483
0.201270688616 0.00264256066446 0.246904312469 0.062574160849 0.486608270344
0.0351638726863 0.00134537491697 0.495420779691 0.466158046924 0.00191192298617
0.109657831909 0.0121946482412 0.244363459657 0.630289356399 0.00349470210552
0.577526133508 0.0348446889063 0.168112710343 0.211886522515 0.00762994509527
0.0259217142629 0.055547719399 0.0130987843278 0.902963267307 0.00246851483096
0.344343332038 0.484053416978 0.00901009626318 0.158974790204 0.00361836772438
0.10338664284 0.170521017798 0.0138133642697 0.471988378203 0.240290592451
0.263902003426 0.0686754240863 0.0280176990278 0.0668555985178 0.57254927202
0.577834578687 0.0599198560022 0.0770621925659 0.275754072952 0.00942930075643
0.101946916439 0.381924520687 0.00258750879716 0.263535999616 0.250005054509
0.0972372809867 0.490836079127 0.000453432499832 0.0632909933949 0.348182212577
0.730846309345 0.189223079431 0.0170034984199 0.0594744591316 0.00345265752714
0.730705690154 0.0141568021967 0.19530025148 0.0528934394582 0.00694381581844
0.12563362004 0.428336607536 0.00915868535702 0.433746346401 0.00312474114086
0.0411325165195 0.70089570979 0.00124945365113 0.254684260203 0.00203806095021
0.14153989371 0.00178429120236 0.730267109966 0.123695146971 0.00271355930952
0.242346687388 0.59750699386 0.00507575233812 0.151483963741 0.00358660260049
0.0153951291731 7.75221767422e-05 0.923211725329 0.060503397021 0.000812226584355
0.0271574461073 0.000209114318539 0.873961081655 0.0978452044601 0.000827155340252
0.0373333474495 0.000556677051516 0.759520399564 0.201066907643 0.00152266804667
0.144478422008 0.116692811829 0.0221682946699 0.28226071262 0.434399756499
0.300040203729 0.0164245080601 0.344052564144 0.333896365977 0.0055863566387
0.235333135382 0.00130504060035 0.719106883885 0.0415065183597 0.00274842088943
0.387980157257 0.0279362531234 0.250085123443 0.327832804165 0.00616566493602
0.434341892703 0.265448357144 0.0309178686877 0.265837806578 0.00345407718698
0.728496551338 0.0322576020185 0.108780243584 0.12109220856 0.00937339326419
0.279451025065 0.29809585942 0.00437667763771 0.0966172882229 0.321459159317
0.810520626579 0.0187949007834 0.117701227071 0.045367287873 0.00761595909765
0.0362203039991 0.000139999327692 0.40954990277 0.0402498920134 0.513839893289
0.0308261503977 0.0160602948822 0.0359441336029 0.348999827593 0.568169589708
0.117323043401 0.00633639675897 0.415272255199 0.457768444873 0.00329986175839
0.478280361892 0.0240748431706 0.261669956022 0.229173107876 0.00680173003702
0.0106110707878 0.0136300328494 0.0182544004674 0.955948185718 0.00155631000015
0.413038185054 0.0501598009649 0.0555884381431 0.0794045331069 0.401809047063
0.396112180113 0.00605644794856 0.271890727113 0.0875046851653 0.23843595767
0.193137917352 0.0206354662889 0.245616620972 0.536536768415 0.00407322923138
0.208358917091 0.00997918356635 0.415794548524 0.361722992032 0.00414435934618
0.0183208808709 3.49358462402e-05 0.958128009027 0.0219535213883 0.0015626514422
0.308071302547 0.0648902455881 0.0734503786854 0.303839809099 0.249748267146
0.0799068440699 0.669523718524 0.000306931419505 0.0656923111716 0.184570200233
0.00220778206097 0.000229241985788 0.00124205620064 0.00170316499567 0.994617740929
0.261377082639 0.0562286169123 0.0790823729584 0.597001212116 0.00631071406706
0.0343042340485 0.0215373182178 0.0386191815362 0.478906110543 0.426633156809
0.0393449958053 0.522506940972 0.00228072251682 0.43382717522 0.00204016711156
0.0415591806721 0.000109875470061 0.684005934601 0.0397381698486 0.234586837431
0.223993832041 0.150920299682 0.0459716445069 0.576174620747 0.00293960226765
0.0288824422898 0.000177136554537 0.166961110707 0.0283392770253 0.775640031843
0.542845261043 0.256590419428 0.0258283986576 0.170829900147 0.00390602220643
0.0345967940597 0.00103943349103 0.567337329989 0.395312267121 0.00171417566142
0.11030623131 0.00306225259141 0.59170144434 0.292166455263 0.00276361535063
0.018992964463 0.00580745216648 0.0149373188311 0.0764001892333 0.883862076375
0.0342683126431 8.55374768959e-05 0.550946648266 0.0330761036718 0.381623388342
0.168182674912 0.000776549130474 0.791901970722 0.037072519066 0.00206628563508
0.319612737682 0.160917069642 0.0516154599994 0.464564627768 0.00329010836459
0.113793928348 0.0208700217788 0.15606761675 0.705843369102 0.00342506477206
0.13547687218 0.333554732975 0.0139561795295 0.514170808251 0.00284140734527
0.0266617087314 0.00598251681426 0.111484763663 0.85403828469 0.00183272700994
0.165439308552 0.0295463800734 0.0281089090883 0.0529885370025 0.72391686311
0.203648768585 0.0143820026291 0.330020738495 0.447862691237 0.00408579796484
0.0325517784465 0.00258703902579 0.290912310848 0.671881450094 0.00206742191677
0.883326348033 0.0375054934419 0.0393747907021 0.0329299936205 0.00686337455589
0.498039418089 0.0708247091855 0.0717131405794 0.350921629675 0.00850110321137
0.444035428219 0.485294081889 0.00507437960359 0.0630635429856 0.00253256633397
0.0964744569309 0.864042492906 0.000425311874228 0.0348742708555 0.00418346829598
0.185515369644 0.0314760432304 0.159061190438 0.619568317499 0.00437908052289
0.0703779142774 0.0368722317122 0.0453492206103 0.844552553496 0.00284807927005
0.349360450437 0.0418113606034 0.162773331283 0.440144431025 0.0059104289611
0.23131964767 0.254164468795 0.0295597539652 0.482329949509 0.00262618181084
0.0342273442815 0.924021272769 0.000148895754754 0.0397455558838 0.00185692984931
0.1949580729 0.00657642461658 0.504977165808 0.289815597367 0.00367273847206
0.0370264457797 0.186046854971 0.00744311048456 0.767560239326 0.00192334730276
0.412885759183 0.0712821394057 0.0720871550046 0.436300945174 0.00744400442322
0.337738202271 0.374449088345 0.0180769742895 0.266580670783 0.00315506379946
0.648461057425 0.181527028613 0.0316140456079 0.134181797606 0.00421606887453
0.445820206304 0.370216746394 0.0152836795922 0.165376599776 0.00330276861137
0.606592411188 0.13185215479 0.0446773482678 0.211229645313 0.00564844376637
0.0370956741661 0.00411372816854 0.226467397134 0.729982040458 0.00234115788493
0.0326316931732 0.000621041120706 0.699908516928 0.265213054773 0.00162569216191
0.0522201638121 0.00227328707575 0.229844613396 0.265563308519 0.450098631063
0.0457790727589 0.0012693684024 0.159136101212 0.15086959206 0.642945867017
0.292673693168 0.00948717360633 0.453253935245 0.239647843288 0.0049373551765
0.221066203437 0.00312636055564 0.649078375528 0.122997177514 0.00373188414917
0.107740702546 0.000408265237584 0.857576831467 0.0329194095729 0.00135479306521
0.651110571922 0.0264168417233 0.180804883311 0.133848781814 0.00781892364094
0.0313247954862 0.000739783402153 0.636452325529 0.329930496532 0.00155259925644
0.0393985259382 0.429390998579 0.00310077264996 0.525887747184 0.00222195526358
0.660329427712 0.0510203436558 0.0798847273305 0.19969561539 0.00906988462778
0.0165111735207 0.000105770918898 0.0844695306006 0.0183943779239 0.880519140496
0.0473252884624 0.0119201885466 0.00877731434607 0.0204186458745 0.911558562237
0.0830557120929 0.021458956459 0.104724040228 0.788066992177 0.00269430197345
0.205195711072 0.0013242439939 0.481927813334 0.0621632658574 0.24938896193
0.542300224682 0.378857147371 0.00818268874616 0.0681290802507 0.00253085487991
0.114346880534 0.00448183230763 0.500729922322 0.377558758492 0.00288260713641
0.0264849699582 3.52740389985e-05 0.828536349043 0.0211322502289 0.123811146546
0.612754548649 0.0340764537616 0.0721637236321 0.088744732704 0.192260539659
0.393579393166 0.00329251105389 0.548616550873 0.0506465848185 0.00386495586474
0.564799128601 0.00696161642841 0.368907303422 0.0539667290845 0.00536522440848
0.56517711888 0.018035408858 0.268885860909 0.140178085493 0.00772352505813
0.328332860593 0.265829682431 0.0322046685316 0.370313983457 0.00331880483958
0.302799417931 0.00541888218269 0.549034548048 0.138368330366 0.00437882229792
0.0236526132644 0.00927345769077 0.0583170189556 0.906373112214 0.00238379849968
0.155069962359 0.786314354711 0.00103101107848 0.0551033559328 0.00248131473802
0.288565257725 0.025714739548 0.253034034521 0.427425881361 0.00526008912569
0.024279635639 0.0263228029361 0.0308974625326 0.767433541453 0.15106656192
0.140451297715 0.706959014958 0.00259468089855 0.147811344087 0.00218366033034
0.416519003468 0.165661525548 0.0515966400732 0.36262090874 0.00360192258137
0.0690308778457 0.167241800926 0.000629549662855 0.0469869359741 0.716110835211
0.241867373619 0.487307533629 0.0100149928035 0.257779886004 0.0030302170846
0.722048565731 0.0903315705832 0.0416035205658 0.138730931959 0.00728541083154
0.129510910805 0.151131735849 0.0309637968816 0.685565979974 0.00282757588679
0.0418562361599 0.258015774153 0.00614071102885 0.691481512962 0.00250576792978
0.140478668434 0.0466485749656 0.0638566356781 0.745107111939 0.00390900769438
0.266159188972 0.038549360452 0.165956529281 0.524071383303 0.00526353931762
20 changes: 20 additions & 0 deletions Dataset/Yahoo/YahooKMeansModel/10kmeans_model20.dat
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
0.0317079870503 0.0332996698323 0.0429205187449 0.888496517046 0.00357530751892
0.305401710928 0.00392088096683 0.582670008398 0.0989051952597 0.00910220402389
0.0117228579645 0.00279260456798 0.00776988502612 0.0112922063243 0.966422436705
0.547686397585 0.0127973993141 0.320710263666 0.109322307422 0.00948363238884
0.0485917084622 0.904658496384 0.000281013523708 0.0437442815053 0.00272448504032
0.0873578386424 0.201451965691 0.0172119450937 0.686453467613 0.00752478329288
0.0198047994921 5.53290953953e-05 0.954169670706 0.0245243946088 0.00144578897704
0.0933313971948 0.0198974696752 0.15845505077 0.716486698943 0.0118293832816
0.0895382622479 0.054070856966 0.101583647659 0.148803926456 0.606003305337
0.121471928952 0.419113981609 0.00926083839304 0.442383817153 0.00776943445801
0.501353560838 0.312298600433 0.0186380770621 0.161292686896 0.00641707489133
0.291075256509 0.572000512207 0.00509776253555 0.11673708579 0.0150893844098
0.311630836884 0.017274392451 0.338927434244 0.324836462924 0.00733087351023
0.304436251622 0.0902475680548 0.0995768182809 0.497103684908 0.00863567781739
0.53211615074 0.0759837587631 0.101055851234 0.280850719926 0.00999352008407
0.082053626276 0.00580865483729 0.368535258283 0.538399625764 0.00520283503278
0.763187438963 0.0655457208927 0.073942836823 0.0882627132013 0.0090612907974
0.0889980467042 0.000747910435874 0.788130139509 0.110155302553 0.0119686007638
0.0788079910459 0.684500371198 0.00202229773408 0.22245046671 0.0122188731378
0.0765806718491 0.00223496083719 0.594598400426 0.321143205511 0.00544276072182
40 changes: 40 additions & 0 deletions Dataset/Yahoo/YahooKMeansModel/10kmeans_model40.dat
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
0.0569255732776 0.0116196709427 0.139612623248 0.78785879791 0.00398333447878
0.182387019921 0.00399521367471 0.604154914507 0.204423778945 0.00503907354675
0.00674265363757 0.00112910876034 0.00390834809812 0.00587194952049 0.982347929412
0.537118682167 0.0683682260638 0.0961340066016 0.289025518703 0.00935356760887
0.154797387824 0.77158747413 0.00118819311159 0.0624354902147 0.00999145612612
0.0772386028186 0.248674032197 0.0120737755464 0.658507648867 0.00350594095601
0.0528592676438 0.000277741413346 0.871696085763 0.0691318706494 0.00603503483658
0.0667684307338 0.00358149203163 0.425154959773 0.50088880225 0.00360631518465
0.261869505064 0.0716492387645 0.0526838203459 0.090559460105 0.523237974903
0.0658217949121 0.412781571042 0.00557663625361 0.511933282485 0.00388671581848
0.453408215083 0.417713133922 0.00993236810212 0.114890491875 0.00405578991622
0.281233433712 0.577785529026 0.0052090004431 0.130615310269 0.00515672830235
0.424600328853 0.0258783586277 0.255255917694 0.286251167887 0.00801422782397
0.0644789386403 0.00658413262349 0.27324604997 0.65247901513 0.00321186307112
0.449072864203 0.23495873253 0.0347448613972 0.275889729223 0.00533381234662
0.259137989124 0.0108072129224 0.421257023685 0.302921029027 0.00587674542741
0.681596784719 0.0635303187263 0.0788081140307 0.164845559094 0.011219223205
0.335974808978 0.00371458673946 0.575936312799 0.0789514644422 0.00542282611767
0.254689527988 0.317533545689 0.0232934978707 0.399171187653 0.00531224116727
0.0567702903609 0.00175541890012 0.582167974507 0.356281593357 0.00302472204327
0.0296880292206 0.934683896649 0.000139042740935 0.0339724799278 0.00151653117612
0.0542703669247 0.748147083572 0.00124831831045 0.19200880709 0.00432542377442
0.36374482914 0.0825463352631 0.0917096212983 0.454983323325 0.00701589332611
0.477213468176 0.00852515191914 0.40578409452 0.100915594627 0.00756169132007
0.233703742936 0.0226021488524 0.254615203666 0.483727229983 0.00535167554977
0.0457641933279 0.000717831664391 0.74707021266 0.203273488652 0.00317427280867
0.0689163574857 0.102279282464 0.0246395100118 0.797556102813 0.00660874616478
0.111785565559 0.388286831237 0.00122425728023 0.0994931300678 0.399210216087
0.0513587537237 0.000341913739394 0.433741766737 0.0562795027137 0.458278056323
0.204223964781 0.0699397515182 0.0879248072896 0.632826391388 0.00508508537438
0.637418013232 0.0161550591199 0.240185979756 0.0969318669193 0.00930908103912
0.0223305152371 0.0229936474139 0.0353822722089 0.916630895879 0.00266266963451
0.0550622568029 0.0394151338085 0.0567680076136 0.491155241075 0.357599359607
0.643037959516 0.230489248372 0.0200354567606 0.102180706181 0.00425662944107
0.820359061127 0.0522703588513 0.0617304117392 0.0575967567973 0.00804341273014
0.182759406865 0.00123443208402 0.750722173528 0.0587831719215 0.00650081569446
0.0544917786905 0.0202081481266 0.0476534310533 0.0540014325258 0.823645208984
0.0865411465838 0.567680418705 0.0039268726564 0.337427623687 0.00442393946813
0.0492145976201 0.0277509181376 0.0521583892277 0.231798071386 0.639078022322
0.0127015973091 2.35844256425e-05 0.970372861618 0.0160063446418 0.000895589914049
Loading