Filter 在elasticsearch中插入时在logstash中将字符串解析为日期

我的csv文件中的一条记录如下所示

,No,FMN1116CD,Holiday Order,2782427,Mr Vijay Ran ,9/6/2014,17/11/2014,,To Deliver,0,S,FALSE,2726149,-1,-1,,0,,,,-1,Delhi Inbound,NEW DELHI,Basic Hotel Order Details,Custom Package,Double,,Others,25500,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,890,0,0,3280,0,29670,,,FALSE,not paid,287894747,,TRUE,,5,-301767,0,50764,,,,,,,,,,,,-3065174,,,,,,,,,,,,,,,,,,,,2,India,22/11/2014,,Manual,Custom Package,26889,Balasore,Callcenter,0,0,0,0

我的conf文件看起来像

input {  
  file {
      path => "/home/sagnik/Work/logstash-1.4.2/bin/ho.csv"
      start_position => "beginning"
  }
}



filter {
    date {
       match => ["Travel_Date", "dd/MM/YYYY"]

    }
    csv {
       columns =>     ["Comm_Plan","Queue_Booking","Order_Reference","Multi_Ordertype","Order_Item_Id","Pax_Name","Generation_Date","Travel_Date","Desk_ID","Status","SalesID","UserRole","Group_Booking","Agent_ID","Admin_ID","Partner_ID","Partner_Name","AgencyAdmin_Id","Supp_Pmt_Ref","Supp_Pmt_Acc","Supp_Pmt_Status","Distributor","Agent_Name","State","Supplier_Code","Secondary_Supplier_Code","Supplier_Number","PNR","Ticket_Number","Basic","Taxes","OCTax","Meal_Price","Cab_Price","Handling","PLB","Deposit_Incentive","Subagent_Handling","Subagent_Plb","Subagent_Deposit_Incentive","Dist_Comm","Stax_Air","Booking_Surcharge","TDS","SubAgent_TDS","Dist_TDS","Dist_Service_Tax","STax_Bas","Partner_Booking_Fee","Old_Payment_Fee","Transaction_Fee_Rcvd","Transaction_Fee_Givn","Net_Amount","Vouchers","CC","Dist_Credit","Partner_Payment_Status","Call_CenterId","Linked_Order","Is_Holiday","Child_Ordertype","Room_Nights","Payment_Sum","Credit_Outstanding","Payment_Fee","DepositCharge","DepositComm_Cr","CreditCharge","CreditComm_Cr","Distributor_CreditCharge","Distributor_CreditComm_Cr","Vendor_7Charge","CCICICI_MOTO_3DCharge","IPSPCharge","NetBanking_TPSCharge","CCICICI_EMICharge","NetBanking_CITRUSCharge","CCHDFC_MOTOCharge","ACharge","CCAMEXCharge","NetBanking_4Charge","NetBanking_PayUCharge","Ccivrscharge","Vch_LossVoucher","Vch_StaffTravel","Vch_DiscountB2C","Vch_ViaPointsRedemption","Vch_DealVoucher","Vch_BonusRedemption","Vch_Loss","Vch_MultiOrder","Vch_SME","Vch_TripCard","Vch_NetPayments","Vch_OfferPromo","Vch_HotelPromotion","No_Of_Pax","Hotel_CountryName","Checkout_Date","Hotel_Booking_Code","Hotel_Type","Hotel_Name","Hotel_Id","Hotel_City","Hotel_Booked_By","Hotel_Net","Hotel_Taxes","Hotel_Gross","Hotel_Supplier_Commission"]
       separator => ","
    }
}



output {  
    elasticsearch {
        host => "localhost"
        index => "hotel"
    }
    stdout { codec => rubydebug }
}

---

但是在插入之后，

Travel\u Date

以字符串形式出现，而不是日期。因此，我无法用它进行任何导航。请帮助我，我想你误解了日期

date

插件用于解析字段值并将其与

@timestamp

字段匹配

如果需要将字段值从字符串转换为日期，可以使用

Ruby

plugin来实现。 通过下面的配置，我可以解析

Travel\u Date

到Date格式，并在elasticsearch中导航它。 看一看

input {  
  file {
      path => "/home/sagnik/Work/logstash-1.4.2/bin/ho.csv"
      start_position => "beginning"
  }
}



filter {

    csv {
       columns =>     ["Comm_Plan","Queue_Booking","Order_Reference","Multi_Ordertype","Order_Item_Id","Pax_Name","Generation_Date","Travel_Date","Desk_ID","Status","SalesID","UserRole","Group_Booking","Agent_ID","Admin_ID","Partner_ID","Partner_Name","AgencyAdmin_Id","Supp_Pmt_Ref","Supp_Pmt_Acc","Supp_Pmt_Status","Distributor","Agent_Name","State","Supplier_Code","Secondary_Supplier_Code","Supplier_Number","PNR","Ticket_Number","Basic","Taxes","OCTax","Meal_Price","Cab_Price","Handling","PLB","Deposit_Incentive","Subagent_Handling","Subagent_Plb","Subagent_Deposit_Incentive","Dist_Comm","Stax_Air","Booking_Surcharge","TDS","SubAgent_TDS","Dist_TDS","Dist_Service_Tax","STax_Bas","Partner_Booking_Fee","Old_Payment_Fee","Transaction_Fee_Rcvd","Transaction_Fee_Givn","Net_Amount","Vouchers","CC","Dist_Credit","Partner_Payment_Status","Call_CenterId","Linked_Order","Is_Holiday","Child_Ordertype","Room_Nights","Payment_Sum","Credit_Outstanding","Payment_Fee","DepositCharge","DepositComm_Cr","CreditCharge","CreditComm_Cr","Distributor_CreditCharge","Distributor_CreditComm_Cr","Vendor_7Charge","CCICICI_MOTO_3DCharge","IPSPCharge","NetBanking_TPSCharge","CCICICI_EMICharge","NetBanking_CITRUSCharge","CCHDFC_MOTOCharge","ACharge","CCAMEXCharge","NetBanking_4Charge","NetBanking_PayUCharge","Ccivrscharge","Vch_LossVoucher","Vch_StaffTravel","Vch_DiscountB2C","Vch_ViaPointsRedemption","Vch_DealVoucher","Vch_BonusRedemption","Vch_Loss","Vch_MultiOrder","Vch_SME","Vch_TripCard","Vch_NetPayments","Vch_OfferPromo","Vch_HotelPromotion","No_Of_Pax","Hotel_CountryName","Checkout_Date","Hotel_Booking_Code","Hotel_Type","Hotel_Name","Hotel_Id","Hotel_City","Hotel_Booked_By","Hotel_Net","Hotel_Taxes","Hotel_Gross","Hotel_Supplier_Commission"]
        separator => ","
    }
    ruby {
            code => "
                    event['Travel_Date'] = Date.parse(event['Travel_Date']);
            "
    }
}



output {  
    elasticsearch {
        host => "localhost"
        index => "hotel"
    }
    stdout { codec => rubydebug }
}

---

希望这能对您有所帮助。

您需要在本案例中使用target

 date {
       match => ["Travel_Date", "dd/MM/YYYY"]
        target => "New_Field_Name"
    }

---

嘿@BenLim，有没有类似的方法可以传递整数、浮点和其他数据库？你能给我一个我可以查的链接吗？到目前为止我还没有试过。也许你可以试试。我想鲁比能做到！