using string.strip() in python to extract specific coloumns

Question

import requests
from bs4 import BeautifulSoup
f = open('path to create /Price.csv','w')
errorFile = open('path to create /errorPrice.txt','w')
year = 2012; month = 1; day =1
if year<= 2016:
    if day > 32:
        month += 1
        day = 1
    if month >12:
        year += 1
        month = 1
    url = 'http://nepalstock.com.np/main/todays_price/index/1/stock-name/desc/YTozOntzOjk6InN0YXJ0RGF0ZSI7czoxMDoiMjAxNi0wNi0wOSI7czoxMjoic3RvY2stc3ltYm9sIjtzOjA6IiI7czo2OiJfbGltaXQiO3M6MjoiNTAiO30?startDate='+str(year)+'-'+str(month)+'-'+str(day)+'&stock-symbol=&_limit=500'
    res = requests.get(url)

    soup = BeautifulSoup(res.text, 'lxml')

    for child in soup.findAll('table'):
            for row in child.findAll('tr')[2:]:
                for col in row.findAll('td'):

                    try:
                        SN = col[3].string.strip()
                        f.write(SN+'\n')

                    except Exception as e:
                        errorFile.write (str(day) + '*************'+ str(e)+'***********************'+ str(col)+'\n')
                        pass
        #day += 1
f.close
errorFile.close

'I wanted to extract col[3] but it wouldn't work and shows nothing I can traceback in error file although I am a complete noob and maybe mistaken on that bit.'

@Saad Yes!! , and Moses I have no idea what the difference is but just using strip didn't work either. — mad, Jun 13 '16 at 12:09

Simon Kirsten · Answer 1 · 2016-06-13T12:29:04.090

A few things about your code before your actual error:

Use the with statement to open files. Manually opening and closing files is unnecessary.

Use res.content instead of res.text if you do not plan on printing the web page. If you are passing the page source to another function like soup.parse always use res.content.

About your problem:

row.findAll('td') is the list of all the table data, from wich you need the 3d index, so you do not need to iterate over it.

Just use it like this:

for child in soup.findAll('table'):
    for row in child.findAll('tr')[2:-4]:
        cols = row.findAll('td')
        SN = cols[3].string.strip()
        print(SN)

Also, as you can see by the -4 the last 4 rows do also not contain any data.

score 1 · Accepted Answer · edited May 23 '17 at 10:33

You just need to index the list returned SN = row.findAll('td')[3], there is also no point removing the newline to then put it back on, this is will get all the table data:

 r = requests.get("http://nepalstock.com.np/main/todays_price/index/1/stock-name/desc/YTozOntzOjk6InN0YXJ0RGF0ZSI7czoxMDoiMjAxNi0wNi0wOSI7czoxMjoic3RvY2stc3ltYm9sIjtzOjA6IiI7czo2OiJfbGltaXQiO3M6MjoiNTAiO30?startDate=2016-01-04&stock-symbol=&_limit=500")

soup = BeautifulSoup(r.content)

table = soup.select_one("table.table-condensed.table-hover")
cols_tr = table.select_one("tr.unique")
print([td.text.strip() for td in cols_tr.find_all("td")])

for row in cols_tr.find_all_next("tr"):
    print([td.text.strip() for td in row.find_all("td")])

Which will output:

[u'S.N.', u'Traded Companies', u'No. Of Transaction', u'Max Price', u'Min Price', u'Closing Price', u'Traded Shares', u'Amount', u'Previous Closing', u'Difference Rs.']
[u'1', u'Yeti  Development Bank Limited', u'7', u'143.00', u'140.00', u'140.00', u'1777.00', u'252897.00', u'143.00', u'-3.00']
[u'2', u'Western Development Bank Limited', u'1', u'315.00', u'315.00', u'315.00', u'500.00', u'157500.00', u'319.00', u'-4.00']
[u'3', u'Vijaya laghubitta Bittiya Sanstha Ltd.', u'162', u'1235.00', u'1172.00', u'1195.00', u'3188.00', u'3853176.00', u'1225.00', u'-30.00']
[u'4', u'United Insurance Co. (Nepal) Ltd.', u'111', u'550.00', u'500.00', u'540.00', u'50433.00', u'26399127.00', u'510.00', u'30.00']
[u'5', u'Unique Finance Ltd.', u'6', u'147.00', u'145.00', u'147.00', u'280.00', u'41116.00', u'143.00', u'4.00']
[u'6', u'Uniliver Nepal Limited', u'1', u'25500.00', u'25500.00', u'25500.00', u'80.00', u'2040000.00', u'25186.00', u'314.00']
[u'7', u'Tourism Development Bank Limited', u'2', u'233.00', u'229.00', u'229.00', u'51.00', u'11719.00', u'229.00', u'0.00']
[u'8', u'Tinau Development Bank Limited', u'1', u'300.00', u'300.00', u'300.00', u'154.00', u'46200.00', u'300.00', u'0.00']
[u'9', u'Taragaon Regency Hotel Limited', u'4', u'221.00', u'213.00', u'216.00', u'1550.00', u'334550.00', u'217.00', u'-1.00']
[u'10', u'Synergy Finance Ltd.', u'6', u'100.00', u'97.00', u'100.00', u'7991.00', u'793118.00', u'98.00', u'2.00']
[u'11', u'Swarojgar Laghu Bitta Bikas Bank Ltd.', u'9', u'2040.00', u'1921.00', u'2025.00', u'932.00', u'1859391.00', u'2000.00', u'25.00']
[u'12', u'Swabalamban Bikas Bank Limited', u'36', u'2025.00', u'1940.00', u'1940.00', u'3024.00', u'5970152.00', u'1949.00', u'-9.00']
[u'13', u'Surya Life Insurance Company Limited', u'27', u'826.00', u'773.00', u'790.00', u'5354.00', u'4322799.00', u'804.00', u'-14.00']
[u'14', u'Sunrise Bank Limited', u'7', u'415.00', u'408.00', u'415.00', u'2162.00', u'891160.00', u'412.00', u'3.00']
[u'15', u'Summit Micro Finance Development Bank Ltd.', u'4', u'2404.00', u'2311.00', u'2356.00', u'128.00', u'301158.00', u'2266.00', u'90.00']
[u'16', u'Subhechha Bikas Bank Limited', u'13', u'319.00', u'312.00', u'319.00', u'1130.00', u'356240.00', u'310.00', u'9.00']
[u'17', u'Standard Chartered Bank Limited', u'12', u'2590.00', u'2550.00', u'2555.00', u'805.00', u'2069500.00', u'2630.00', u'-75.00']
[u'18', u'Soaltee Hotel Limited', u'24', u'325.00', u'310.00', u'310.00', u'12580.00', u'3947660.00', u'320.00', u'-10.00']
[u'19', u'Siddhartha Investment Growth Scheme-1', u'1', u'15.56', u'15.56', u'15.56', u'1500.00', u'23340.00', u'15.61', u'-0.05']
[u'20', u'Siddhartha Insurance Ltd.', u'57', u'1075.00', u'1011.00', u'1075.00', u'10540.00', u'11093865.00', u'1052.00', u'23.00']
[u'21', u'Siddhartha Equity Orineted Scheme', u'4', u'9.55', u'9.50', u'9.55', u'7600.00', u'72527.00', u'9.65', u'-0.10']
[u'22', u'Siddhartha Bank Limited', u'20', u'605.00', u'591.00', u'597.00', u'4844.00', u'2904232.00', u'610.00', u'-13.00']
[u'23', u'Siddharth Finance Ltd.', u'8', u'196.00', u'180.00', u'193.00', u'677.00', u'128312.00', u'179.00', u'14.00']
[u'24', u'Shikhar Insurance Co. Ltd.', u'42', u'1424.00', u'1345.00', u'1379.00', u'6993.00', u'9595832.00', u'1377.00', u'2.00']
[u'25', u'Shangrila Development Bank Ltd.', u'3', u'380.00', u'375.00', u'378.00', u'657.00', u'248160.00', u'380.00', u'-2.00']
[u'26', u'Sewa Bikas Bank Limited', u'2', u'260.00', u'260.00', u'260.00', u'304.00', u'79040.00', u'255.00', u'5.00']
[u'27', u'Sanima Mai Hydropower Ltd.', u'33', u'768.00', u'750.00', u'760.00', u'2373.00', u'1808638.00', u'765.00', u'-5.00']
[u'28', u'Sanima Bank Limited', u'33', u'660.00', u'653.00', u'653.00', u'11727.00', u'7700213.00', u'661.00', u'-8.00']
[u'29', u'Sana Kisan Bikas Bank Ltd', u'37', u'1456.00', u'1400.00', u'1434.00', u'2863.00', u'4077092.00', u'1400.00', u'34.00']
[u'30', u'Sahayogi Bikas Bank Limited', u'1', u'375.00', u'375.00', u'375.00', u'10.00', u'3750.00', u'368.00', u'7.00']
[u'31', u'Sahara Bikas Bank Ltd.', u'2', u'980.00', u'980.00', u'980.00', u'20.00', u'19600.00', u'1000.00', u'-20.00']
[u'32', u'Sagarmatha Insurance Co. Ltd.', u'18', u'1122.00', u'1075.00', u'1075.00', u'2179.00', u'2389443.00', u'1095.00', u'-20.00']
[u'33', u'Sagarmatha  Finance Limited', u'20', u'185.00', u'170.00', u'185.00', u'3484.00', u'629111.00', u'169.00', u'16.00']
[u'34', u'Rural Microfinance Development Centre Ltd.', u'15', u'690.00', u'680.00', u'686.00', u'1490.00', u'1016830.00', u'689.00', u'-3.00']
[u'35', u'Ridi Hydropower Development Company Ltd.', u'34', u'362.00', u'343.00', u'355.00', u'648.00', u'228537.00', u'346.00', u'9.00']
[u'36', u'Reliance Lotus Finance Ltd.', u'1', u'170.00', u'170.00', u'170.00', u'1000.00', u'170000.00', u'170.00', u'0.00']
[u'37', u'Reliable Microfinance Bittiya Sanstha Ltd.', u'1', u'327.00', u'327.00', u'327.00', u'10.00', u'3270.00', u'321.00', u'6.00']
[u'38', u'Reliable Development Bank Limited', u'11', u'300.00', u'295.00', u'295.00', u'482.00', u'143941.00', u'300.00', u'-5.00']
[u'39', u'Raptibheri Bikas Bank Ltd.', u'68', u'185.00', u'177.00', u'182.00', u'1406.00', u'252988.00', u'180.00', u'2.00']
[u'40', u'Purnima Bikas Bank Ltd.', u'4', u'318.00', u'300.00', u'318.00', u'404.00', u'124866.00', u'298.00', u'20.00']
[u'41', u'Prudential Insurance Co. Ltd.', u'113', u'720.00', u'630.00', u'698.00', u'31521.00', u'21716588.00', u'655.00', u'43.00']
[u'42', u'ProgressiveFinance Limited', u'2', u'126.00', u'126.00', u'126.00', u'240.00', u'30240.00', u'128.00', u'-2.00']
[u'43', u'Professional Diyalo Bikas Bank Ltd.', u'1', u'165.00', u'165.00', u'165.00', u'11.00', u'1815.00', u'164.00', u'1.00']
[u'44', u'Prime Life Insurance Company Limited', u'89', u'1595.00', u'1550.00', u'1565.00', u'12795.00', u'20032091.00', u'1555.00', u'10.00']
[u'45', u'Prime Commercial Bank Ltd.', u'19', u'499.00', u'485.00', u'490.00', u'4224.00', u'2064184.00', u'490.00', u'0.00']
[u'46', u'Premier Insurance Co. Ltd.', u'120', u'965.00', u'895.00', u'960.00', u'26299.00', u'24644627.00', u'878.00', u'82.00']
[u'47', u'Prabhu Insurance Ltd.', u'60', u'629.00', u'590.00', u'605.00', u'17659.00', u'10658300.00', u'617.00', u'-12.00']
[u'48', u'Pokhara Finance Ltd.', u'6', u'292.00', u'285.00', u'290.00', u'1616.00', u'463864.00', u'289.00', u'1.00']
[u'49', u'Pacific Development Bank Limited', u'1', u'333.00', u'333.00', u'333.00', u'53.00', u'17649.00', u'339.00', u'-6.00']
[u'50', u'Oriental Hotels Limited', u'1', u'417.00', u'417.00', u'417.00', u'22.00', u'9174.00', u'416.00', u'1.00']
[u'51', u'NMB Sulav Investment Fund-1', u'2', u'9.90', u'9.71', u'9.90', u'3500.00', u'34365.00', u'9.90', u'0.00']
[u'52', u'NMB Bank Limited', u'12', u'424.00', u'416.00', u'421.00', u'2559.00', u'1078416.00', u'416.00', u'5.00']
[u'53', u'NLG Insurance Company Ltd.', u'38', u'891.00', u'806.00', u'842.00', u'7364.00', u'6196657.00', u'825.00', u'17.00']
[u'54', u'Nirdhan Utthan Bank Limited', u'8', u'1690.00', u'1670.00', u'1688.00', u'996.00', u'1672610.00', u'1655.00', u'33.00']
[u'55', u'NIDC Capital Markets Ltd.', u'1', u'276.00', u'276.00', u'276.00', u'500.00', u'138000.00', u'281.00', u'-5.00']
[u'56', u'NIC Asia Bank Ltd.', u'168', u'1034.00', u'1010.00', u'1012.00', u'41061.00', u'41629557.00', u'1014.00', u'-2.00']
[u'57', u'NIBL Samriddhi Fund 1', u'7', u'9.95', u'9.90', u'9.95', u'20478.00', u'203256.10', u'9.80', u'0.15']
[u'58', u'Nerude Laghubita Bikas Bank Limited', u'18', u'2038.00', u'1977.00', u'1985.00', u'1880.00', u'3750102.00', u'1960.00', u'25.00']
[u'59', u'Nepal SBI Bank Limited', u'35', u'1510.00', u'1500.00', u'1500.00', u'7476.00', u'11234652.00', u'1503.00', u'-3.00']
[u'60', u'Nepal Life Insurance Co. Ltd.', u'14', u'3520.00', u'3410.00', u'3461.00', u'1661.00', u'5730070.00', u'3475.00', u'-14.00']
[u'61', u'Nepal Investment Bank Ltd. Promoter Share', u'6', u'591.00', u'585.00', u'591.00', u'3460.00', u'2039850.00', u'600.00', u'-9.00']
[u'62', u'Nepal Investment Bank Limited', u'37', u'700.00', u'692.00', u'700.00', u'9062.00', u'6311182.00', u'695.00', u'5.00']
[u'63', u'Nepal Insurance Co. Ltd.', u'23', u'618.00', u'579.00', u'615.00', u'7147.00', u'4295495.00', u'617.00', u'-2.00']
[u'64', u'Nepal Grameen Bikas Bank  Ltd.', u'25', u'313.00', u'299.00', u'300.00', u'9497.00', u'2881799.00', u'314.00', u'-14.00']
[u'65', u'Nepal Doorsanchar Comapany Limited', u'2', u'660.00', u'659.00', u'660.00', u'1320.00', u'871000.00', u'660.00', u'0.00']
[u'66', u'Nepal Bank Limited', u'29', u'306.00', u'300.00', u'301.00', u'23894.00', u'7224247.00', u'310.00', u'-9.00']
[u'67', u'Nepal Bangladesh Bank Limited', u'49', u'436.00', u'430.00', u'436.00', u'17799.00', u'7713258.00', u'432.00', u'4.00']
[u'68', u'Neco Insurance Co. Ltd.', u'50', u'890.00', u'832.00', u'864.00', u'8940.00', u'7675605.00', u'868.00', u'-4.00']
[u'69', u'Naya Nepal Laghubitta Bikas Bank Ltd.', u'6', u'1475.00', u'1400.00', u'1410.00', u'1173.00', u'1663779.00', u'1497.00', u'-87.00']
[u'70', u'National Life Insurance Co. Ltd.', u'66', u'2630.00', u'2500.00', u'2526.00', u'9181.00', u'23332004.00', u'2550.00', u'-24.00']
[u'71', u'National Hydro Power Company Limited', u'31', u'115.00', u'110.00', u'112.00', u'34550.00', u'3886400.00', u'110.00', u'2.00']
[u'72', u'Narayani National Finance Limited', u'2', u'295.00', u'294.00', u'295.00', u'652.00', u'192140.00', u'295.00', u'0.00']
[u'73', u'NagBeli LaghuBitta Bikas Bank Ltd.', u'7', u'3570.00', u'3500.00', u'3500.00', u'303.00', u'1071390.00', u'3550.00', u'-50.00']
[u'74', u'NABIL Bank Limited Promotor Share', u'8', u'1341.00', u'1332.00', u'1332.00', u'1897.00', u'2534803.00', u'1340.00', u'-8.00']
[u'75', u'Nabil Bank Limited', u'28', u'1850.00', u'1800.00', u'1800.00', u'2496.00', u'4504646.00', u'1840.00', u'-40.00']
[u'76', u'Nabil Balance Fund 1', u'2', u'14.50', u'14.40', u'14.40', u'4797.00', u'69256.50', u'14.70', u'-0.30']
[u'77', u'Multipurpose Finance Company  Limited', u'3', u'150.00', u'150.00', u'150.00', u'33.00', u'4950.00', u'150.00', u'0.00']
[u'78', u'Muktinath Bikas Bank Ltd.', u'19', u'720.00', u'690.00', u'701.00', u'4694.00', u'3281696.00', u'720.00', u'-19.00']
[u'79', u'Mount Makalu Development Bank Ltd.', u'1', u'506.00', u'506.00', u'506.00', u'20.00', u'10120.00', u'497.00', u'9.00']
[u'80', u'Mithila LaghuBitta Bikas Bank Ltd.', u'1', u'1322.00', u'1322.00', u'1322.00', u'24.00', u'31728.00', u'1297.00', u'25.00']
[u'81', u'Miteri Development Bank Limited', u'11', u'555.00', u'535.00', u'550.00', u'3168.00', u'1726818.00', u'535.00', u'15.00']
[u'82', u'Mirmire Microfinance Development Bank Ltd.', u'84', u'2310.00', u'2150.00', u'2225.00', u'840.00', u'1877350.00', u'2280.00', u'-55.00']
[u'83', u'Malika Bikas Bank Limited', u'3', u'247.00', u'245.00', u'245.00', u'850.00', u'208350.00', u'250.00', u'-5.00']
[u'84', u'Mahila Sahayatra Microfinance Bittiya Sanstha Ltd.', u'4', u'552.00', u'522.00', u'552.00', u'40.00', u'21480.00', u'512.00', u'40.00']
[u'85', u'Lumbini General Insurance Co. Ltd.', u'55', u'670.00', u'575.00', u'610.00', u'7968.00', u'4860101.00', u'610.00', u'0.00']
[u'86', u'Life Insurance Co. Nepal', u'25', u'3877.00', u'3780.00', u'3798.00', u'2958.00', u'11241460.00', u'3872.00', u'-74.00']
[u'87', u'Laxmi Value Fund-1', u'1', u'9.95', u'9.95', u'9.95', u'3300.00', u'32835.00', u'9.94', u'0.01']
[u'88', u'Laxmi Laghubitta Bittiya Sanstha Ltd.', u'22', u'1899.00', u'1757.00', u'1800.00', u'1010.00', u'1864729.00', u'1826.00', u'-26.00']
[u'89', u'Laxmi Bank Limited', u'36', u'572.00', u'555.00', u'566.00', u'8151.00', u'4587286.00', u'569.00', u'-3.00']
[u'90', u'Kumari Bank Limited', u'21', u'399.00', u'390.00', u'392.00', u'6652.00', u'2609547.00', u'398.00', u'-6.00']
[u'91', u'Kasthamandap Development Bank Limited', u'4', u'183.00', u'180.00', u'183.00', u'287.00', u'51780.00', u'180.00', u'3.00']
[u'92', u'Kalinchowk Development Bank Ltd.', u'22', u'174.00', u'166.00', u'166.00', u'1123.00', u'189964.00', u'174.00', u'-8.00']
[u'93', u'Kalika Microcredit Development Bank Ltd.', u'19', u'1645.00', u'1582.00', u'1600.00', u'2594.00', u'4149688.00', u'1614.00', u'-14.00']
[u'94', u'Kabeli Bikas Bank Limited', u'5', u'469.00', u'453.00', u'460.00', u'124.00', u'56536.00', u'453.00', u'7.00']
[u'95', u'Jebils Finance Ltd.', u'1', u'156.00', u'156.00', u'156.00', u'100.00', u'15600.00', u'153.00', u'3.00']
[u'96', u'Innovative Development Bank Ltd.', u'3', u'540.00', u'520.00', u'540.00', u'30.00', u'15900.00', u'510.00', u'30.00']
[u'97', u'ILFCO Microfinance Bittiya Sanstha Ltd.', u'200', u'1035.00', u'980.00', u'1000.00', u'3140.00', u'3169471.00', u'1030.00', u'-30.00']
[u'98', u'ICFC Finance Limited', u'1', u'204.00', u'204.00', u'204.00', u'500.00', u'102000.00', u'204.00', u'0.00']
[u'99', u'Himalayan General Insurance Co. Ltd', u'154', u'671.00', u'600.00', u'671.00', u'57596.00', u'36967536.00', u'610.00', u'61.00']
[u'100', u'Himalayan Bank Limited', u'13', u'932.00', u'915.00', u'926.00', u'1030.00', u'954487.00', u'932.00', u'-6.00']
[u'101', u'Gurans Life Insurance Company Ltd.', u'46', u'728.00', u'661.00', u'686.00', u'12487.00', u'8592354.00', u'700.00', u'-14.00']
[u'102', u'Global IME Bank Limited', u'49', u'396.00', u'392.00', u'394.00', u'12643.00', u'4985295.00', u'395.00', u'-1.00']
[u'103', u'Garima Bikas Bank Limited', u'5', u'299.00', u'296.00', u'299.00', u'404.00', u'120036.00', u'301.00', u'-2.00']
[u'104', u'Gandaki Bikas Bank Limited', u'3', u'336.00', u'325.00', u'336.00', u'295.00', u'98375.00', u'321.00', u'15.00']
[u'105', u'First Micro Finance Development Bank Ltd.', u'15', u'722.00', u'709.00', u'709.00', u'3001.00', u'2149232.00', u'730.00', u'-21.00']
[u'106', u'Excel Development Bank Ltd.', u'19', u'754.00', u'731.00', u'741.00', u'2834.00', u'2104352.00', u'734.00', u'7.00']
[u'107', u'Everest Insurance Co. Ltd.', u'31', u'1026.00', u'930.00', u'951.00', u'4349.00', u'4225220.00', u'999.00', u'-48.00']
[u'108', u'Everest Bank Limited Con. Pref.', u'6', u'1340.00', u'1320.00', u'1320.00', u'1372.00', u'1815640.00', u'1349.00', u'-29.00']
[u'109', u'Everest Bank Limited', u'27', u'2055.00', u'2036.00', u'2050.00', u'4501.00', u'9209523.00', u'2041.00', u'9.00']
[u'110', u'Dev Bikas Bank Limited', u'8', u'181.00', u'175.00', u'175.00', u'3328.00', u'590568.00', u'184.00', u'-9.00']
[u'111', u'Deprosc Development Bank Limited', u'21', u'1820.00', u'1725.00', u'1735.00', u'6895.00', u'12041655.00', u'1759.00', u'-24.00']
[u'112', u'Country Development Bank Ltd.', u'3', u'180.00', u'180.00', u'180.00', u'180.00', u'32400.00', u'180.00', u'0.00']
[u'113', u'Civil Bank Ltd', u'51', u'255.00', u'247.00', u'250.00', u'14794.00', u'3707676.00', u'252.00', u'-2.00']
[u'114', u'Citizen Investment Trust', u'21', u'4150.00', u'4100.00', u'4135.00', u'1517.00', u'6252870.00', u'4105.00', u'30.00']
[u'115', u'Citizen Bank International Limited', u'29', u'567.00', u'553.00', u'563.00', u'5521.00', u'3109845.00', u'568.00', u'-5.00']
[u'116', u'Chilime Hydropower Company Limited', u'40', u'1175.00', u'1150.00', u'1150.00', u'4385.00', u'5083376.00', u'1166.00', u'-16.00']
[u'117', u'Chhimek Laghubitta Bikas Bank Limited', u'27', u'1775.00', u'1680.00', u'1680.00', u'2320.00', u'4030330.00', u'1755.00', u'-75.00']
[u'118', u'Century Commercial Bank Ltd.', u'27', u'316.00', u'308.00', u'314.00', u'7663.00', u'2395955.00', u'314.00', u'0.00']
[u'119', u'Butwal Power Company Limited', u'4', u'540.00', u'540.00', u'540.00', u'700.00', u'378000.00', u'540.00', u'0.00']
[u'120', u'Biratlaxmi Bikas Bank Limited', u'5', u'315.00', u'303.00', u'315.00', u'332.00', u'102712.00', u'309.00', u'6.00']
[u'121', u'Bhaktapur Finance Co. Ltd.', u'12', u'140.00', u'133.00', u'133.00', u'1321.00', u'177546.00', u'140.00', u'-7.00']
[u'122', u'Barun Hydropower Co. Ltd.', u'102', u'280.00', u'266.00', u'276.00', u'1050.00', u'286600.00', u'267.00', u'9.00']
[u'123', u'Asian Life Insurance Co. Limited', u'45', u'1300.00', u'1240.00', u'1275.00', u'9473.00', u'12147812.00', u'1304.00', u'-29.00']
[u'124', u'Arun Valley Hydropower Development Co. Ltd.', u'23', u'261.00', u'254.00', u'257.00', u'5072.00', u'1296773.00', u'256.00', u'1.00']
[u'125', u'Arun Finance Limited', u'10', u'110.00', u'104.00', u'104.00', u'1550.00', u'162080.00', u'112.00', u'-8.00']
[u'126', u'Araniko Development Bank Limited', u'5', u'208.00', u'202.00', u'207.00', u'1079.00', u'222403.00', u'199.00', u'8.00']
[u'127', u'Api Power Company Ltd.', u'449', u'373.00', u'358.00', u'368.00', u'9183.00', u'3351712.00', u'369.00', u'-1.00']
[u'128', u'Agriculture Development Bank Limited', u'37', u'528.00', u'502.00', u'506.00', u'6760.00', u'3441749.00', u'518.00', u'-12.00']
[u'129', u'Ace Development Bank Ltd.', u'4', u'290.00', u'288.00', u'290.00', u'1368.00', u'395744.00', u'290.00', u'0.00']
[u'Page 1/1\xa01']
[u'Total Amount Rs.', u'494,473,367']
[u'Total Quantity', u'704,022']
[u'Total No of Transactions', u'3,766']

To avoid the last data that is not part of the cols, catch where the first td is not a digit:

for row in cols_tr.find_all_next("tr"):
    if row.find("td").text.isdigit():
         print([td.text.strip() for td in row.find_all("td")])

That will leave you with just the table data, removing:

[u'Page 1/1\xa01']
[u'Total Amount Rs.', u'494,473,367']
[u'Total Quantity', u'704,022']
[u'Total No of Transactions', u'3,766']

You could also check the length of the tds returned to see if they match the length of the columns. What you don't want to do is hardcode slices as if there was an extra row added before the header or after the last piece of data you would lose.

Also if you just want the nth td, you don't need find_all, you can use nth-of-type:

for row in cols_tr.find_all_next("tr"):
    if row.find("td").text.isdigit():
         print(row.select_one("td:nth-of-type(4)").text)

That will just give you the fourth column data:

143.00
315.00
1235.00
550.00
147.00
25500.00
233.00
300.00
221.00
100.00
2040.00
2025.00
826.00
415.00
2404.00
319.00
2590.00
325.00
15.56
1075.00
9.55
605.00
196.00
1424.00
380.00
260.00
768.00
660.00
1456.00
375.00
980.00
1122.00
185.00
690.00
362.00
170.00
327.00
300.00
185.00
318.00
720.00
126.00
165.00
1595.00
499.00
965.00
629.00
292.00
333.00
417.00
9.90
424.00
891.00
1690.00
276.00
1034.00
9.95
2038.00
1510.00
3520.00
591.00
700.00
618.00
313.00
660.00
306.00
436.00
890.00
1475.00
2630.00
115.00
295.00
3570.00
1341.00
1850.00
 ......................................

To write all the data to csv is simply a matter of using a csv.writer: and writing the lists of data:

from bs4 import BeautifulSoup
soup = BeautifulSoup(r.content)
with open("data.csv", "w") as f:

    wr = writer(f)
    table = soup.select_one("table.table-condensed.table-hover")
    cols_tr = table.select_one("tr.unique")
    # write columns names
    wr.writerow([td.text.strip() for td in cols_tr.find_all("td")])

    # write all row data.
    wr.writerows([td.text.strip() for td in row.find_all("td")]
                    for row in cols_tr.find_all_next("tr")
                        if row.find("td").text.isdigit())

So data.csv will look like:

S.N.,Traded Companies,No. Of Transaction,Max Price,Min Price,Closing Price,Traded Shares,Amount,Previous Closing,Difference Rs.
1,Yeti  Development Bank Limited,7,143.00,140.00,140.00,1777.00,252897.00,143.00,-3.00
2,Western Development Bank Limited,1,315.00,315.00,315.00,500.00,157500.00,319.00,-4.00
3,Vijaya laghubitta Bittiya Sanstha Ltd.,162,1235.00,1172.00,1195.00,3188.00,3853176.00,1225.00,-30.00
4,United Insurance Co. (Nepal) Ltd.,111,550.00,500.00,540.00,50433.00,26399127.00,510.00,30.00
5,Unique Finance Ltd.,6,147.00,145.00,147.00,280.00,41116.00,143.00,4.00
6,Uniliver Nepal Limited,1,25500.00,25500.00,25500.00,80.00,2040000.00,25186.00,314.00
7,Tourism Development Bank Limited,2,233.00,229.00,229.00,51.00,11719.00,229.00,0.00
8,Tinau Development Bank Limited,1,300.00,300.00,300.00,154.00,46200.00,300.00,0.00
9,Taragaon Regency Hotel Limited,4,221.00,213.00,216.00,1550.00,334550.00,217.00,-1.00
10,Synergy Finance Ltd.,6,100.00,97.00,100.00,7991.00,793118.00,98.00,2.00
11,Swarojgar Laghu Bitta Bikas Bank Ltd.,9,2040.00,1921.00,2025.00,932.00,1859391.00,2000.00,25.00
12,Swabalamban Bikas Bank Limited,36,2025.00,1940.00,1940.00,3024.00,5970152.00,1949.00,-9.00
13,Surya Life Insurance Company Limited,27,826.00,773.00,790.00,5354.00,4322799.00,804.00,-14.00
14,Sunrise Bank Limited,7,415.00,408.00,415.00,2162.00,891160.00,412.00,3.00
......................................

To iterate over a range of dates, use the logic here

I understand it a lot better now. But still struggling to extract that first output you provided into a csv file. — mad, Jun 13 '16 at 16:50
to extract those output data with same structure into a csv file. — mad, Jun 14 '16 at 09:34

using string.strip() in python to extract specific coloumns

2 Answers2