sql - Vb.net rounding inconcistancy -
this question has answer here:
- real vs. floating point vs. money 4 answers
i using vb.net summarize information in datatable using following code.
for reason rounding works on value apart 1.
code:
private sub button4_click(sender object, e eventargs) handles button4.click if textbox1.text = "" 'do nothing else dim conn new sqlclient.sqlconnection conn .connectionstring = "server=inlt01\sqlexpress; database=orbisbilling; integrated security=yes" .open() end dim cmd new sqlclient.sqlcommand cmd .commandtext = "select [account], [customer lookup],round (sum ([chargedamount]),2) [sum of buy price],round (sum ([sell price]),2) [sum of sell price],[tariff lookup] [orbisbilling].[dbo].[" + textbox1.text + "] group [account], [customer lookup],[tariff lookup] order [customer lookup]" .commandtype = commandtype.text .commandtimeout = 30 .connection = conn end dim dt new datatable dt.load(cmd.executereader) orbisviewbillform.datagridview1 .autogeneratecolumns = true .datasource = dt end cmd.dispose() cmd = nothing conn.dispose() conn = nothing orbisviewbillform.show() end if end sub
as can see round sum 'sell price' 2 decimal places.
here result.
56.1 1.5899999999999999 133.64 12.67 14.93 0.76 0.57 4.25 2.24 0.06 7.75 4.31 7.96 0.78 0.01 33.55
if rounding 2, why 1.5899999999999999?
to troubleshoot changed rounding 3 , following inconstancies.
56.1 1.5899999999999999 133.643 12.671 14.928 0.764 0.568 4.253 2.2439999999999998 0.058 7.745 4.306 7.9559999999999995 0.779 0.005 33.554
any appreciated.
thanks,
the sql server float
, real
data types don't store decimal values exactly.
you should consider changing database columns decimal or numeric.
Comments
Post a Comment