Older Version
Newer Version
scapegoat
Jan 21, 2006
- "Removed surplus branch label"
**Text file duplicate line remover** [[user:scapegoat]] This code demonstrates the use of line input to read whole lines individually from a text file. [[code]][setupMainWindow]nomainwin WindowWidth = 550 WindowHeight = 410 UpperLeftX=int((DisplayWidth-WindowWidth)/2) UpperLeftY=int((DisplayHeight-WindowHeight)/2) BackgroundColor$ = "buttonface" ForegroundColor$ = "black" statictext #Main.SourcefileLabel, "File to open:", 5, 15, 73, 20 statictext #Main.DestfileLabel, "Output file:", 5, 60, 89, 20 textbox #Main.SourcefilePath, 105, 12, 315, 25 textbox #Main.DestfilePath, 105, 57, 315, 25 button #Main.InFinder, "Browse", [BrowseInPath], UL, 440, 12, 80, 25 button #Main.OutFinder, "Browse", [BrowseOutPath], UL, 440, 58, 80, 25 statictext #Main.TexteditorLabel, "File preview:", 5, 122, 75, 20 texteditor #Main.textedit5, 105, 122, 415, 170 button #Main.Pv, "Preview file", [Preview], UL, 105, 317, 195, 25 button #Main.Go, "Remove duplicates", [Clean], UL, 330, 317, 190, 25 open "Duplicate Line Remover" for window as #Main print #Main, "font ms_sans_serif 0 16" print #Main, "trapclose [quit]" outpath$ = "" OriginalFilePath$ = "" print #Main.DestfilePath, outpath$ print #Main.SourcefilePath, OriginalFilePath$ [null] wait [BrowseInPath] filedialog "Open text file", "*.txt", OriginalFilePath$ print #Main.SourcefilePath, OriginalFilePath$ wait [BrowseOutPath] filedialog "Open text file", "*.txt", outpath$ print #Main.DestfilePath, outpath$ wait [Preview] print #Main.SourcefilePath, "!contents? OriginalFilePath$"; if OriginalFilePath$ = "" then goto [null] open OriginalFilePath$ for input as #rm while eof(#rm)=0 line input #rm, txt$ print #Main.textedit5, txt$ wend close #rm wait [Clean] originalentries=0 quantityofduplicates=0 print #Main.DestfilePath, "!contents? outpath$"; print #Main.SourcefilePath, "!contents? OriginalFilePath$"; if (outpath$ = "" or OriginalFilePath$ = "") then goto [null] OPEN outpath$ for output as #writefile close #writefile print #Main.textedit5, "Created output file:" print #Main.textedit5, outpath$ open OriginalFilePath$ for input as #rm while eof(#rm)=0 line input #rm, txt$ originalentries=originalentries+1 duplicate=0 OPEN outpath$ for input as #writefile while eof(#writefile)=0 line input #writefile, compare$ if txt$=compare$ then duplicate=1 : quantityofduplicates=quantityofduplicates+1 wend close #writefile OPEN outpath$ for append as #writefile write$ = txt$ if duplicate=0 then print #writefile, write$ if duplicate=0 then print #Main.textedit5, write$ close #writefile wend close #rm print #Main.textedit5, " " print #Main.textedit5, "Done. The original list contained "_ +str$(quantityofduplicates)+" duplicate lines," print #Main.textedit5, "out of "+str$(originalentries)+" originals. "_ +str$(originalentries-quantityofduplicates)_ +" lines were unique." print #Main.textedit5, "The cleaned file has been saved at:" print #Main.textedit5, outpath$ wait [quit] close #Main end[[code]]