« first day (13 days earlier)      last day (4945 days later) » 

7:00 PM
Huh?
Those are the methods await and async translations use, right?
 
yeah
 
Oh, you're talking about the new tagline?
I only just saw it
 
for the new room description to work, the "CSharp5" class would have to be an odd one
yeah
 
Your comment did seem strangely out of context :P
 
(I like the tagline)
(but it's an odd class ;))
var future = await Task.Factory.StartNew( () => return new CSharp5() );
but it's not as pretty sounding
though:
var future = await GetCSharp5();
 
7:03 PM
if the class has the right methods, it can be used like that
 
at least that's probably more realistic, as a method returning a Task<T> is common
yeah - as I said, you'd have to make a class that implements those methods ;) It's just an odd use case, but definitely possible
 
Well, in a certain way it makes sense for C# 5 to implement GetAwaiter() and such :P
Since that is what it'll end up doing
 
Yeah, I thought it made perfect sense
 
As I said - I like it ;) I just don't think people will write much code where you await on a constructor
 
7:10 PM
I certainly hope not
 
probably not, but it made the keyword new fit nicely, don't you think? :)
 
yeah
var future = await (new CSharp5()).UseAsync();
 
I wonder what they'll be doing for the next Visual Studio
IntelliTrace and those parallel debugging things were pretty big
 
@Joren ChuckNorris.CreateWebsiteForTheClient(clientRequirements);
2
 
A ChuckNorris library would be a .NET Framework feature, not a VS feature :P
 
7:26 PM
Hi everyone
 
@LasseVKarlsen when did I become familiar enough with this stuff that reading his blog makes total sense and I understand the topics he's discussing? I'm not sure to be scared or not
 
Enjoy the moment, it'll soon pass :)
It always does, in my experience :P
 
I'm new to C# and I have some questions about large DataTables.
 
Welcome @Angelina ... what can we help you do? Or can you post it to a SO question and paste that URL here?
 
7:30 PM
@drachenstern Maybe you should go read Eric Lippert's series on CPS then :P
 
@Joren Yeah, that'll bring him down to earth real quick :)
 
One second, I'll create a question for good measure.
 
@Angelina If the questions are likely to have definitive answers, SO itself is the best place to ask them
 
@Joren I started to but I had some db integration work to work on, and decided that chatroom and staring at SQL was enough fun for one simultaneous session
I take it you guys do mean this series starting with ? blogs.msdn.com/b/ericlippert/archive/2010/10/21/…
 
Yeah
Other good ones are about lambda type inference :D
And some of the Co- and contravariance posts
Talking about posts with a nice bit of complexity to them, of course
 
7:35 PM
yeah, I understand and follow the logic when I read those, but I've not tried writing that code in C# as I just do ASP.NET during the day and on the weekends, well, let's just say that I don't have the free weekends that I want. But I'm well able to follow along with what he discusses and I understand not just the why but the how
 
Yeah
 
@LasseVKarlsen, I don't know if there is a definitive answer.
0
Q: How should I manage large DataTables?

AngelinaHi everyone, For reasons that don't make a lot of sense (Read: Not my decision) I need to keep a large number of rows, about ~90,000, in a DataTable and I do not have the option of using a database. I need to be able to search the DataTable efficiently to find rows that match some basic criteri...

There we go.
 
@Angelina What other limitations do you have? .NET 2.0 for instance? Are you allowed to use LINQ?
 
Not able to use LINQ. I'm using VS2010 and I believe that is .NET 4.0? I'm trying to find the version in VS2010; I grabbed VS only about a month ago if that is any indicator.
 
Why can't you use a database?
 
7:44 PM
How come you can't use LINQ if you have .NET 4.0?
 
It has been requested that I do not. As I mentioned, that is not my decision and I think that it is a very poor decision. I've also been asked not to use LINQ. Nobody has explained to me the details of why.
 
requested by whom?
 
So, they give you restrictions ... just out of spite? :P
 
This is largely why I've found this tricky. @LasseVKarlsen, I don't want to get into that. It is a headache.
 
@Angelina you can try EF 4
 
7:46 PM
Ok, do you have any idea why you can't use LINQ?
Because, honestly, it's like saying "You can't use IF-statements"
 
@Incognito if she cant use LINQ or a database for ideological reasons its doubtful she'll be able to use EF...
 
"ideological"? You mean "stupid"?
 
@LasseVKarlsen i thought they were synonyms?
 
I've also been asked not to use any frameworks or libraries outside of what's been provided to me out of the box with C#.
 
@entens she has no real reason not to use LINQ. Anyway if it is restricted why not EF
 
7:48 PM
Except LINQ
 
Does the restriction on LINQ apply to not using iterators?
 
Which of course is provided out of the box with C# (and .NET)
 
I just took a look at EF, very interesting. All of this will be useful to me in the future in better situations.
 
Lol so typical of SO people to get hung on on questioning the questioner rather than solve the problem... Honestly, does it matter who asked Angelina not to use a database?
 
You could implement a basic subset of LINQ yourself (Select and such is not too complicated) and then use that
 
7:49 PM
No
It matters why
Because it might be other rules we need to know about
 
Or just consider how you would write things in LINQ and do that with for loops :/
 
For instance, if I say:
Do this:
x.Where(...)
 
@LasseVKarlsen that's why I asked
 
Then I'm using LINQ
 
@gmagana so typical of programmers to focus only on the immediate solution and not the problem domain.
 
7:50 PM
I've given you the information that I have. I have to try and make the best of this situation. I don't like it either.
 
@entens If the problem domain is not asked about, then it's not the problem to solve
 
anyway if you can't use LINQ, EF .... then you have to iterate through DataSet yourself to find the needed value
 
@Angelina then in the absence of LINQ I would suggest a foreach over the rows in the datatable and add them to a new datatable and return the new
that's about the fastest way to do it
 
Build dictionaries, basically implement the index system a database would give you
 
@gmagana but if we can find a faster way to do it then we want to
 
7:51 PM
If you need to quickly find all columns that has the value X in column Y, build a dictionary for column Y
 
@LasseVKarlsen but how does she store that as a simple element? She'll need to build a wrapper class for her datatable at that point, no?
 
Why?
She loads the datatable, builds the dictionary, and is ready to go
 
@LasseVKarlsen keeping things in sync
 
Does the datatable change? (the question doesn't say)
 
we weren't told
I presume so, eventually
@Angelina ... well?
 
7:52 PM
No, I'm sorry
If you say: Only consider what I tell you, then I will
 
The DataTable does not change.
 
Then build dictionaries
 
ok, then in that case build dictionaries ;)
 
All of the data is pulled from a flat file and then does not change.
 
eh. might be overcomplicating the problem in the first place. If we are all talking about System.Data.DataTable, then it implements a LINQ free Select method already.
 
7:54 PM
Oh, and by the way, get a new job
 
What is even the point of using a DataTable if you're only using it as an intermediate to build dictionaries out of :/
 
Select on an unindexed column is a loop
 
@LasseVKarlsen doesn't mean you cant implement an index before preforming the Select()
 
@entens yeah, but we were looking for ways to help her speed things up and be more efficient, and I have a feeling the maintenance programmers would do better with a more thorough foreach ;)
@LasseVKarlsen Not so polite, eh? If I were to count the times I've heard that sort of comment :(
 
Well, I'm sorry, but if you're given arbitrary limitations, like "Don't use LINQ", without justification and reason, then there really is no reason not to get a new job
 
7:57 PM
@drachenstern I don't see that as a personal attack, really
 
And it doesn't really if you have justification or reason
 
Or "don't use var" (Hi all)
 
@Joren she may not have the resources or the means to just up and quit her job
 
As I said, it's like being told: "don't use IF-statements"
 
@LasseVKarlsen I agree but still may be well out of her control
 
7:58 PM
I didn't say she should do it now
 
hi @Benjol
lol, true @LasseVKarlsen
 
It would be appreciated if we could just focus on the project. As I've already indicated, I'm not enjoying this either. Not every problem has seemingly arbitrary limitations like this, though, so let's just leave the nature of what my job 'is' or 'is not' out of the picture.
 
@drachenstern Well, of course. You should take it as 'consider getting ...' :)
 
And I am serious
@Angelina OK, I'll agree with that, but what about building dictionaries for the columns you need to query on? Is that a viable option?
or... let me ask a different question
Will you only ask like this: WHERE col = value
 
@Angelina yah but of course. So your options seem to be either DataTable.Select() (see the answer on your question) or to work with indexes/dictionaries and a foreach. Do you feel comfortable with those options?
 
7:59 PM
Or do you need to ask like this as well: WHERE col BETWEEN value1 AND value2?
 
@LasseVKarlsen she said that "where = 2" would be her search
 
Dictionaries are fine; and yes - I will only ever be asking WHERE col = value.
 
@Lasse: Isn't that just a case of using SortedDictionary instead?
 
@drachenstern She didn't say that would be her only search
@Angelina Then I would go with dictionaries
 
@LasseVKarlsen in the question she said so, but she did say over two columns
 
8:01 PM
Something like this:
var colxLookup = new Dictionary<DataTypeForColumnX, DataRow>();
and then populate it
or rather
 
/nods
 
var colxLookup = new Dictionary<DataTypeForColumnX, List<DataRow>>();
if you're expecting some values to occur more than once
To be honest, I'm not a big fan of DataTable
It's like using an excel spreadsheet when you really need a database
Let me rephrase that: I don't use DataTable all that much, for all I know there's indexing capabilities you can use without building a dictionary
 
Basically a complete answer would have to be "write your own in-memory database system that uses the DataTable as basic storage" ... that makes me very sad
The only relief being that the kind of queries you get is limited, so you don't have to implement everything
 
@Joren or alternately, use the DataTable.Select
 
And of course the data being static
 
8:06 PM
My main problem with the whole ... problem ... is "do not use LINQ"
I still wonder why that limitation was placed
And ... to be honest ... what it means
 
@drach: Well, of course, if you don't mind the 90k loop
 
For instance, the .ToDictionary(...) extension method was added as part of the whole LINQ release
 
@LasseVKarlsen yeah, I've run across it a few times, it means "I'm scared of LINQ and I expect it to be a performance nightmare so I don't want to introduce it into our codebase"
 
Is it LINQ? ... I don't know
What it actually means is "I don't know what LINQ is"
and to be honest, I don't know what scares me more
 
@Joren she's gonna have to at some point, no?
 
8:07 PM
True
 
You can do the 90K loop once, and build dictionaries
 
@LasseVKarlsen micromanagement FTL
 
or whatever
 
it all gets compiled to IL anyways. let the compiler figure it out.
 
@entens of course, and the better answer for now
 
8:08 PM
Personally, I would do one of two things
 
let @Angelina build it, then let her optimize it based on feedback, and we'll still be around, yeah?
 
1. Build sorted lists of the "stuff" I want to look up, I would pick this if I need to do ranged lookups, ie. X BETWEEN a AND b
2. Build dictionaries of key -> row (or list of rows)
If I were to wager on the reasoning behind the "No LINQ" limitation, I would gather it would be a fear that all "queries" would end up as "table scans"
 
@Lasse: How would you handle the slightly more complex queries (where x = 3 and y = 5). Lookup '3' in your x-dict, 5 in your y-dict, then loop over the results and cross-reference?
 
ie. scan the whole 90K dataset
 
Or you could build a special xy-dictionary for that ...
Where does it end :P
 
8:10 PM
@Joren If I were forced to do this without a proper database, I would build a dictionary for X, and one for Y, do the lookup on both, and correlate the results
 
Isn't to index or not to index the more complicated queries the kind of thing that databases usually decide to do based on usage statistics and such?
 
@Joren hey now, she just needs to grab Enterprise FizzBuzz for that
 
Now, you could also simply decide to lookup all rows with x=5, loop through them all and pick the ones with y=5
 
@Joren no, that's what you pay a DBA for
 
DBA?
 
8:11 PM
database administrator
 
First I must find all rows that have the same value from Column1. Then, I have to search that subset of rows to find which ones have the desired value of 'x' in Column2. Finally, if there is only one row with 'x' in Column2, the job is done. If there is more than one row with value 'x' in Column2, I must select one of those rows at random.
 
Okay
 
my DBAs regularly watch for performance and implement the indexes
 
Everyone loves a good DBA. I know I sure do. :)
 
@Angelina so a double if on a for loop does the trick?? you just add the rows to a new datatable and spit the new datatable back out
 
8:13 PM
@drachenstern I thought I remembered someone telling me that databases can do that kind of thing automatically nowadays based on usage stats and whatnot
 
@Angelina Here's a tip. Anytime you're given a dataset X, a criteria Y, and the request to simply "from all the rows from X that satisfies criteria Y, pick one at random", always go with the first you find
 
@drachenstern, that seems reasonable. Between your suggestion and the dictionaries idea - which would be best based on the information I just provided (rough high level description.)
@LasseVKarlsen, good idea.
Thank you all, by the way. This is very helpful and I appreciate it. :)
 
DataTable output = new DataTable; foreach (DataRow row in inputDataTable.Rows) { if (row.Column[0] == 2 && row.Column[1] == 2) output.Add(row); } is roughly good enough
 
@Angelina: I'd say, loop once, build the dictionaries while you're doing it so that the second query is faster
 
@Angelina Also, I hope you're not too scared to come back and chat with us later. We are of strong opinions, but we're also open to suggestions :)
 
8:15 PM
aw schucks @Angelina, you've given us something to kill our Friday with ;)
I personally say let her brute force it so they relax on the restrictions :p
 
A slow friday to boot
 
/me playing devils advocate
 
Haha
 
Too scared? Why would I be scared? I like talking to other programmers.
 
Now I am scared!
 
8:16 PM
I'm not scary! Mostly not, as far as I'm aware.
 
lol, because we as a group can be a little overpowering at times ;)
yeah @LasseVKarlsen lost me on that one :S
 
What person in their right mind would like to talk to programmers?
 
Programmers tend to have very strong opinions; they're almost as fierce as political opinions sometimes.
 
@Angelina We're worse
We demand to know the reason behind the decision
 
@Joren, other programmers? We seem to like problems and arguments. @LasseVKarlsen, so very true..
 
8:17 PM
/me also devil's advocate: I love EF, but there are lots of situations where a good Stored Procedure API is a much better approach I think.
 
@Angelina I was joking of course, why else would I be in this chat :)
 
@kervin have you been playing along? she's not got a database :p
 
Well, let me look up the DataTable class, see what it does
 
@Joren lol, I hate talking to other programmers ... gives me all sorts of funky headaches ;)
almost like I wish I could run a GC on some of them ...
 
@drachenstern was just talking in general, didn't read the original question
 
8:18 PM
@kervin yeah I had a feeling ;)
 
@Angelina: So while we're not being completely serious anymore, couldn't you just out of spite implement all the LINQ query operators yourself and just call them by their Haskell names instead of the .NET names? ;)
 
lmao
that's funny
 
Hahaha
 
Then just use them, no one would know :P
 
Hmm, I seem to remember DataTable having some indexing capabilities, why can't I find them?
 
8:21 PM
Oh, Haskell. I've never had anyone ask me to write anything in Haskell. Apparently the comparative programming class I had at university was non-standard and we only looked at ALGOL-like languages; other offerings of the course had functional programming and so I've been kind of toying with the idea of writing something in Haskell or Lisp since then..
 
@Lasse DataView?
 
Looking...
 
It does have some 'reserved for internal use' Index-thing names
 
I think DataView is the one you're referring to, and you can go ahead and supply a "presorted" view to a DataTable so that the underlyings get picked up if changes are made
 
@Angelina Could do F#
 
8:23 PM
@drachenstern Yeah, looking at that, but I don't see how I could defined my own indices
 
screw defining your own, like @entens said, let IL take care of it
why optimize? (until you've proven that you need to)
 
Ah, IBindingList.AddIndex
 
Could do F#; has anyone here ever used F# for anything in production?
 
nope
 
@Angelina Do you know what they meant by "No LINQ"? Did they mean only the LINQ syntax? Or the "loop through everything" part? Or the LINQ extension methods?
 
8:25 PM
Just curious; I don't see a lot of demand for functional programming when it comes to dollars.
 
functional programming will be a big deal going forward with parallel code, but perhaps it won't necessarily be labelled as such
 
Some finance companies apparently have been using it
 
it's big in the science world too
 
No using System.LINQ;
 
8:26 PM
it's just not in demand for LoB apps normally
 
@Angelina Any idea why?
 
@LasseVKarlsen, if I find out, I will let you know.
 
Well, System.LINQ means .NET 3.5
Perhaps they want to be able to compile for and run on .NET 2.0 as well
 
/afk
@LasseVKarlsen that was my original thought (hence why I asked)
 
But it precludes both syntax and extension methods usage
 
8:28 PM
I'm going to go work on this, so if I don't respond it is because I have music on ignoring everything else. :)
 
:)
Let us know how it turns out
I still want you to strongly consider getting a new job, but I also want you to succeed in your current one :) So I want to hear how it went :)
 
8:48 PM
/!afk
 
var t = someEvent;
if (t != null) t();

Do I really need to do this everywhere?
 
@Joren need is a very subjective word...
@Joren if you ever think there's a million in one chance your object will be used in a multi-threaded way... then yes.
 
You may assume that I prefer to avoid NullReferenceExceptions :P
And even without multithreading ...
 
@Joren can I point out that that's how the .NET framework components are coded?
go run reflector on some of their methods, like where you supply Page_Load or such
(and yes, I let my ASP.NET preferences drip out all over the place)
 
Just took a peek, and you're right ... urgh.
I suppose I could attach some silly no-op delegate
 
8:56 PM
wow, the supermultidropdowncomboboxdisplaymechanism-thatIforgotthenameof will show chat @replies too ... interesting
 
Onebox
Or what?
 
no no, the upper left hand box that does the whole network
StackExchangeâ„¢ MultiCollider SuperDropdownâ„¢
 
Oh, that one
Yeah, I don't like that it shows chat replies
But okay
 
9:13 PM
I don't mind that it shows chat replies, I just wish it opened links by default into a new tab
 
Evening folks o/
 
good night :P
 
Question on Interfaces, when I implement an interface on a UserControl I don't get the Name property come across, any reason why?
 
@Sres Probably because there's an existing Name property in the base class. You'll have to add it yourself with an extra 'new' keyword to implement it.
 
I don't understand what either of you mean
What do you mean by "don't get the Name property come across"?
 
9:23 PM
ok, cooling down from the javascript room
why would you bind to an eventhandler that you don't want to respond to?
and then blame the framework?
 
@Pete thanks
@LasseVKarlsen when I implement the interface on the UserControl, then use the smarttag to create all the functions, the Name function wasn't created, @Pete's tip on adding new sorted it though.
 
Ok
To be honest, I really doubt that solved your problem though, but as long as you're happy, I'm OK with it
 
@LasseVKarlsen I managed to correctly guess he was using some tool to generate the code to implement the interface.
@LasseVKarlsen e.g. if you use VS's "Implement interface" command, it will skip properties/method already defined in your class' base class. (unless you do "Explicitly Implement Interface"
 
9:48 PM
ok you lurkers you, @Joren @LasseVKarlsen @rchern @kervin ... anybody feel awake enough to answer the one in chat feedback? chat.stackoverflow.com/transcript/message/63393#63393
 
I don't understand the question
 
@drachenstern Don't quite get what you're asking - the page seems up to date here
it's probably a cache issue
 
Also I have no knowledge about the inner workings of stackoverflow
 
@Joren lol, I figured somebody would have an idea :p ... you guys are lurking in both places ;) ... @ReedCopsey I tried a ctrl-f5, so maybe it was caching issues on SE end and nothing to do with me?
 
what are you seeing?
 
9:51 PM
dead people
 
@ReedCopsey now? the right thing. when I asked? the previous entry for the username.
@Sres close enough?
;)
 
:P
 
my brain is fried enough for one day, so with this, I bid you all adieu
 
10:07 PM
bye
 
10:39 PM
I have a fork-join kind of flow where I run one Task on the thread pool, ContinueWith two tasks, one of which runs on the UI thread, the other on the thread pool, and then ContinueWhenAll on those to do a final bit of work on the UI thread. Is pretty readable and works fine, but I'm itching to try it with async/await to see how much it improves :P
I guess there won't be too much improvements in this case
Since I don't have any exceedingly complex flow control where a state machine would really come in handy
Weird: why would System.Windows.Forms.Form not let me override the Dispose method
 
Dispose is marked with sealed?
 
hmm, from MSDN, no it isn't
Then how :/
Maybe it's the codegen getting in the way?
Yeah, that's it
WinForms designer overrides Dispose for me
 
doh, just looked at the class you were talking about.
ah ye olde myform.designer.cs
 
Yep
 
isitbeeroclock.com ...?
 
10:47 PM
So how do I deal with this ... I can't really edit the Designer.cs file since it'll just be overwritten at some point
 
@Joren: Override protected virtual void Dispose(bool)
that's the standard pattern for IDisposable
Dispose() should always be sealed
 
No, that's what I was saying: 'I' already override Dispose(bool), because myForm.Designer.cs does it
So obviously I can't override it manually a second time
 
oh, yeah, okay
you have to move it to your main class file, and edit it
I always thought they should have added a hook in there for users, though
 
Yeah
I can hook into it by adding something to the components collection that disposes my stuff
But that'd be a weird hack
 
I think the idea would be you'd normally do it on Closing
ie: override OnClosed()
(though the other option is to subscribe to Disposed and do it there, too)
 
10:52 PM
Yeah, but that's both awkward
 
Has anybody else noticed a lot of questions being asked in Chat that really belong as questions on StackOverflow proper?
 
No
@Reed: I'll just move the dispose override to my own code and preserve the old functionality
That seems most sensible
 
yeah - I always worry about tweaking the generated code, but that one's pretty safe
doing it in the OnClosed is I think the "official" way to handle it, though
as close/dispose should always be identical in components that support both
ooo -parent site's down for maintainence :S
 
Again?
 

« first day (13 days earlier)      last day (4945 days later) »