淺談利用邏輯回歸來解決文本分類時(shí)的模型調(diào)優(yōu)

想和數(shù)據(jù)挖掘沾點(diǎn)邊，所以最近在復(fù)習(xí)一些算法，因?yàn)橛謱W(xué)了點(diǎn)R，深感這是個(gè)統(tǒng)計(jì)分析挖掘的利器，所以想用R實(shí)現(xiàn)一些挖掘算法。

樸素貝葉斯法大概是最簡單的一種挖掘算法了，《統(tǒng)計(jì)學(xué)習(xí)方法》在第四章做了很詳細(xì)的敘述，無非是對于輸入特征x，利用通過學(xué)習(xí)得到的模型計(jì)算后驗(yàn)概率分布，將后驗(yàn)概率最大的分類作為輸出。

根據(jù)貝葉斯定理，后驗(yàn)概率P(Y=cx | X=x) = 條件概率P(X=x | Y=cx) * 先驗(yàn)概率P(Y = ck) / P(X=x)，取P(X=x | Y=cx) * P(Y = ck)最大的分類作為輸出。
下面是一個(gè)小數(shù)據(jù)集下使用R進(jìn)行樸素貝葉斯分類的例子,代碼如下：
    #構(gòu)造訓(xùn)練集  
    data <- matrix(c("sunny","hot","high","weak","no",  
                     "sunny","hot","high","strong","no",  
                     "overcast","hot","high","weak","yes",  
                     "rain","mild","high","weak","yes",  
                     "rain","cool","normal","weak","yes",  
                     "rain","cool","normal","strong","no",  
                     "overcast","cool","normal","strong","yes",  
                     "sunny","mild","high","weak","no",  
                     "sunny","cool","normal","weak","yes",  
                     "rain","mild","normal","weak","yes",  
                     "sunny","mild","normal","strong","yes",  
                     "overcast","mild","high","strong","yes",  
                     "overcast","hot","normal","weak","yes",  
                     "rain","mild","high","strong","no"), byrow = TRUE,  
                   dimnames = list(day = c(),  
                   condition = c("outlook","temperature",  
                     "humidity","wind","playtennis")), nrow=14, ncol=5);  
      
    #計(jì)算先驗(yàn)概率  
    prior.yes = sum(data[,5] == "yes") / length(data[,5]);  
    prior.no  = sum(data[,5] == "no")  / length(data[,5]);  
      
    #模型  
    naive.bayes.prediction <- function(condition.vec) {  
        # Calculate unnormlized posterior probability for playtennis = yes.  
        playtennis.yes <-  
            sum((data[,1] == condition.vec[1]) & (data[,5] == "yes")) / sum(data[,5] == "yes") * # P(outlook = f_1 | playtennis = yes)  
            sum((data[,2] == condition.vec[2]) & (data[,5] == "yes")) / sum(data[,5] == "yes") * # P(temperature = f_2 | playtennis = yes)  
            sum((data[,3] == condition.vec[3]) & (data[,5] == "yes")) / sum(data[,5] == "yes") * # P(humidity = f_3 | playtennis = yes)  
            sum((data[,4] == condition.vec[4]) & (data[,5] == "yes")) / sum(data[,5] == "yes") * # P(wind = f_4 | playtennis = yes)  
            prior.yes; # P(playtennis = yes)  
      
        # Calculate unnormlized posterior probability for playtennis = no.  
        playtennis.no <-  
            sum((data[,1] == condition.vec[1]) & (data[,5] == "no"))  / sum(data[,5] == "no")  * # P(outlook = f_1 | playtennis = no)  
            sum((data[,2] == condition.vec[2]) & (data[,5] == "no"))  / sum(data[,5] == "no")  * # P(temperature = f_2 | playtennis = no)  
            sum((data[,3] == condition.vec[3]) & (data[,5] == "no"))  / sum(data[,5] == "no")  * # P(humidity = f_3 | playtennis = no)  
            sum((data[,4] == condition.vec[4]) & (data[,5] == "no"))  / sum(data[,5] == "no")  * # P(wind = f_4 | playtennis = no)  
            prior.no; # P(playtennis = no)  
          
        return(list(post.pr.yes = playtennis.yes,  
                post.pr.no  = playtennis.no,  
                prediction  = ifelse(playtennis.yes >= playtennis.no, "yes", "no")));  
    }  
      
    #預(yù)測  
    naive.bayes.prediction(c("rain",     "hot",  "high",   "strong"));  
    naive.bayes.prediction(c("sunny",    "mild", "normal", "weak"));  
    naive.bayes.prediction(c("overcast", "mild", "normal", "weak")); 
最后一個(gè)分類預(yù)測結(jié)果如下：
$post.pr.yes
[1] 0.05643739

$post.pr.no
[1] 0

$prediction
[1] "yes"