« first day (1564 days earlier)      last day (3381 days later) » 

9:03 AM
Morning
 
@RonniSkansing How insert the array in my mysql db
 
@Poteito same as you would normally? What is the problem you are facing while doing this?
Morning @Fabor
 
@Poteito you have no code writing anything to the DB?
@Fabor o/
 
The problem is i comes from a json file and I would like to insert them in the right records on my db
Here is a vardump of my array codepad.org/kO9tw4jX
You cann see they are not all the same length
Or better question why should I even use json ? What are the advantages from using json
 
9:20 AM
Using json for what? Advantage over what?
 
mornign
 
Morning @PeeHaa o/
 
@PeeHaa moin
 
@nikita2206 JSON for what!
4
 
9:21 AM
lol, give this man a medal :P
 
hehe
I haven't seen the video btw, it's better than thought it would be!
 
@PeeHaa lol
Feeling abit shocked
 
@nikita2206 inorite
 
Symfony vs Laravel.
 
It's pretty fucking hilarious
 
9:26 AM
@JavaFan as you came from Java, CodeIgniter would be much more familiar to you
it has several advantages over both Laravel and Symfony
 
Oh you!
 
it's all JSON and API
 
@SergeyTelshevsky it HAS something like JSON API?
 
it's built around JSON API
with JSON API in mind it is
 
just use the OOP JSON API Framework
 
9:28 AM
thank you @SergeyTelshevsky
 
@RonniSkansing you mean JSOOPI?
 
@PaulCrovella no, the FAPIJO
 
is there a gem for that?
 
posted on January 27, 2015 by kbironneau

/* by BOB808 */

 
@SergeyTelshevsky
 
9:35 AM
@JavaFan
 
JavaFan 1. Do this tutorial -> github.com/PatrickLouys/no-framework-tutorial 2. add the api routes you need, in each route retrieve the content in any way you want, json_encode, set headers and output. 3. profit from your client.
 
WYSIWYG Is available for CodeIgniter?
 
you'll be shocked, guys
Derick doesn't want to remove the warning
 
Good morning
 
mornin'
(I am finally awake & at work)
 
9:39 AM
lol
 
@AndreaFaulds weird
although, the argument of setting UTC as a timezone even if the server is in Asia being stupid kinda rings true to my ears.
 
@PaulCrovella that somehow resonates with the audiobook that I am listening to currently: amazon.com/Supergods-Vigilantes-Miraculous-Mutants-Smallville/…
 
Though I do kinda agree we should keep the warning
There's no good solution unfortunately
 
@RonniSkansing thanks for the link
 
and assuming UTC means people will rely on UTC
Bad.
 
9:45 AM
Rather awesome that netflix has put up shingeki no kyojin for UK people
 
> anime
> Netflix
 
hater ?
 
No, just
Netflix of all things
 
Netflix put up Psycho Pass the other day too. Doing pretty well considering.
 
moin
 
9:58 AM
moin
 
Did anyone of you ever worked with fullcalendar ?
 
@DaveRandom mon amie
 
@AndreaFaulds sans e :P
 
@FlorianMargaine oops :p
But remember, we all thought Dave Chris's son was a girl :p
 
And I thought that Chris' name was Dave
 
10:12 AM
And last name was Random
 
hehehe
 
@PaulCrovella I'd considered something like that
 
Isn't this guy just searching to make use of OR instead of AND, or am I misunderstanding his question.. paste.jesse-obrien.ca/1y31
 
@Duikboot think so
 
:P hmmm
 
@edward please use pastebin or alternative services to show sources here
 
pastebin means
 
@edward just use ->addAttribute('location', 'us');
 
@MarcelBurkhard ok i will try this way
Thanku so much guys
 
10:28 AM
you're welcome
 
hello
Does anyOne have good tutorial for phpexcel ?
i need good tutorial
 
@Bassem The PHPExcel website?
 
^ wuuut, SE has keyboard shortcuts!
 
@HamZa in general or in chat?
 
@MarcelBurkhard general
but you need to enable them
 
10:31 AM
@Bassem and mainly this
 
thanks all
 
mornin'
 
morning... more like lunch time here though
 
@MarcelBurkhard just pretend I woke up really late xD
(it's 7:37 here)
 
10:38 AM
So open to ideas. We have a a bunch of jobs in MySQL. Like millions of rows. We send these jobs to our workers using RabbitMQ. Problem is the time spent for a single script to send out the jobs to the workers and update the MySQL table to say "Sent" exceeds that of the time it takes for the worker to complete them. So our queues keep hitting 0.

Anyone know any good methods for getting jobs sent out faster while keeping the queue relatively low (~1000) but always having something...
 
@FlorianMargaine What's the deal with ami vs. copain? I never used to understand what should be used when (I suck at French)
 
@DaveRandom "ami" = friend, "copain" = buddy
 
to all: probably dumb suggestion, but can't we just parse /etc/sysconfig/clock ZONE entry and use as a sane default for date.timezone?
 
y'know, a friend is someone you can count on. A buddy is just someone you hang out with.
 
@Fabor ...don't use MySQL as a job queue?
 
10:39 AM
@DaveRandom It's not a job queue but a store for which jobs are created from
RabbitMQ is our "queue" so to speak
 
Batch it?
 
@FlorianMargaine ahh I see. I've been told the exact opposite of that before, I always meant ask a French person :-P
 
So process is "Select 1000 rows from MySQL, send 1000 jobs to RabbitMQ, update MySQL with these 1000 IDs to say sent"
Which is too slow.
RabbitMQ doesn't support batch send.
 
It's wrong
 
@Fabor Can you elaborate on that a bit? Meaning that multiple jobs might be created for a given record? What is the data and how is it populated?
@Fabor That sounds like a MySQL-based job queue to me...
 
10:42 AM
Select Job (URL to scrape) from MySQL -> Send via RabbitMQ to worker -> Worker processes
Workers don't know anything about the database
@DaveRandom I'll elaborate a bit on gtalk but we can continue here.
 
anyone experienced in Laravel?
 
@Fabor well one major point I suppose would be that you could probably benefit a lot from using an async rdbms back end
 
@Fabor At least try to do it like this: get some kind of unique token, update a bunch of rows and set this token as a value of some column and at the same time update a field that says this row was sent to rabbit. After update is done SELECT * FROM tbl WHERE token="token" and send this to rabbit
 
zan
Hi guys
 
10:46 AM
@PaulCrovella You've got a fan! I like specially option 2. Then we just create a few standard types on SPL library and problem solved!
 
@nikita2206 That's a good idea.
 
Cause otherwise I guess you were doing UPDATE ... WHERE id IN (1..1000) which can be very slow on large amounts of data
 
@PaulCrovella @AndreaFaulds was referring to this gist.github.com/pcrov/…
 
And also btw don't be afraid to flood rabbit with millions of messages, it's very stable and can handle it pretty easily
 
Yeah RabbitMQ is fine. It's more the rate we can send to rabbit.
Like sending 1000 jobs from a single script takes about a minute.
 
10:52 AM
So the bottleneck is producer -> rabbit interaction or mysql?
 
producer -> rabbit
MySQL selects 1000 jobs fast enough.
 
How do you publish them?
 
basic_publish($message, '', $queue)
I think it's maybe PHP that just isn't fast in this scenario.
 
@Fabor ...did you look at batch_basic_publish and publish_batch at all?
 
Hummm
 
11:00 AM
@Fabor no, after each basic_publich publisher waits for confirmation from rabbit instance. If it's a persistent queue, then for each message it will execute fsync (hitting hard drive)... so you just need to use batch_basic_publish, it will batch messages spending much less time syncing with hard drive
 
@PaulCrovella Hmmm. MY google-fu must be weak. I googled Batch Send RabbitMQ a lot. Cheers will check it out
 
I recommend starting with docs for the library you're using before resorting to google.
 
Sorry :( will do
 
No worries. I'm sure you'll get this worked out quick enough. It's also quite useful to read through the rabbitmq and protocol documentation. When you get a fuller sense how it all works it's a lot easier to solve problems with it.
 
hello guys, can any body help me about how to customize my source codes based on custom parameters ? like they do here: underscores.me..
I want to allow my source code to be customized and generated.. like I have a boiler plate code for starting new PHP project... if anybody wish to use that code then he can customize it, like: prefix, doc comments, project name etc...
 
11:14 AM
blogs.msdn.com/b/ie/archive/2015/01/22/… why can't you just stop making browsers
 
$4,090,277 pledged of $10,000 goal, kickstarter.com/projects/elanlee/exploding-kittens
 
@Gordon someone linked that last week, it was at 2kk back then
 
cats rule the internet
@SergeyTelshevsky and they got 23 days to go
 
The Oatmeal is a cool dude though, so I'm rather fine with that.
 
@Gordon will be at 5kk in the end probably..
 
11:18 AM
@SergeyTelshevsky if they stopped then everyone currently using IE would remain on the current version of IE forever. No one wants that.
 
@Gordon holy fuckeroly :P
 
@PaulCrovella include the setup of any other browser in their new OS?
ship with a lightweight downloader for any of the modern browsers, there are solutions
 
For those hacky git-based deploys, you can use git update-index --assume-unchanged <file> for your tracked config files
 
@SergeyTelshevsky It already does that in the EU. They were forced to make you choose your browser after Win installation.
 
hmm, do I really want to back it with $50 or do I wait until it got mass produced and will be available at amazon at half the price and no shipping?
 
11:21 AM
@Flosi it doesn't help
 
True. It doesn't.
 
they should not include their own browser, make a last LTS and support only that
 
@Gordon Obviously the latter
 
guys please help
 
@Gordon I think they have enough money for you to legitimately be able to do that without feeling bad about it
 
11:24 AM
$encrypt    = md5($password);
There's so much wrong in this line it makes my head hurt
 
@SecondRikudo The biggest issue being the whitespace, obviously.
 
guess I'll wait then
 
@DaveRandom Yeah! There's so much whitespace that it's racist!
 
@SecondRikudo 5f4dcc3b5aa765d61d8327deb882cf99
 
at least it's not a tab
 
11:26 AM
on a completely different topic: I just got beta invited to algorithmia.com. They don't have a PHP client. Anyone interested in porting their Java Api to PHP with me?
 
@Gordon Interested yes, but time nope
 
@PeeHaa that's okay. With the little one around 24/7 I probably have even less time than you. Actually, that's why I am asking :)
 
:)
Should be easy peasy though
AFAICS it's not even oauth but simply passing keys
So it would just be a matter of making requests (unless I am missing something)
 
@AndreaFaulds why did you submit that rfc to reddit with that shitty name? :/
 
@NikiC :D
 
11:29 AM
@PeeHaa yeah, doesnt look too difficult
@NikiC because it's reddit. you have to lower your standards to participate.
 
@AndreaFaulds nothing against a quick jab at derick, but if a title needs a linebreak it is too long ^^
 
Not long enough
 
"She removed the annoying timezone warning from PHP. What happened next blew my mind …"
^ consider this as the headline.
needs more clickbait
 
Damn I just cannot stop staring at that kickstarter campaign...
 
^ buy buy buy
 
11:33 AM
@PeeHaa link?
 
17 mins ago, by Gordon
$4,090,277 pledged of $10,000 goal, https://www.kickstarter.com/projects/elanlee/exploding-kittens
 
@Gordon What she does next is priceless
@PeeHaa Holy crap
 
@Gordon the latter
 
Holy fucking crap indeed
 
@Gordon sry I hadn't scrolled to the bottom and its now fully out of context
 
11:40 AM
Hello everybody. I need DFS recursive function to show elements in array. I did it with stack but I don`t know how to do it with recursion. Can anybody help me?
 
They took meh timezone warning!!1111eleevn
 
"They Took Our Jobs, Took Errr Jerbs, dey turk err jurbz, theeey durrrka duuuuurrr, Durka Durrr."
 
@tereško any info on this ?
 
nope
first time hearing about it
 
this should be fun to watch
 
11:49 AM
@PaulCrovella (popcorn)
 
is it just me, or the whole "use password manager" solution doesn't really solve anything ?
 
oh man I love it, the one upvote he has on SO is answering a question about where the equal sign is
 
guys i need some advice. I've been toying with the idea of breaking down one of my databases and splitting it up into more than one. So reporting is on one database, data is stored somewhere else and then clients is on a seperate one.
is this worthwhile doing or can you not see a point in doing this? Whilst it would cause some refactoring i feel the data has a natural seperation and was wondering on the whys and whereefores of keeping it all in one place.
 
@PaulCrovella lolwut
 
i've come to this thought becauase the client stuff is small, hardly any data, whereas the "Data" stuff is close to 40Gb
 
12:04 PM
@Andy why not in same database?
 
being z a zend_string*, should I get it's value with z->val or should I prefer one of those esoteric macros Zend engine provides?
 
What are you gaining, isnt different tables good enough?
 
40Gb is small as well, I would consider splitting database maybe after a 2xMemory size of DB, still could be optimized if there's a lot of data that is being untouched. Split database is a bitch to work with
 
@RonniSkansing TBH i dont know. i just dont see the point of storing Client info alongside Data. I've had issues with migrations and the like due to the "largeness" of the data that we store./
 
You will have much more problems with migrations if you split your DB
 
12:06 PM
so, it's best to keep the product contained within the same Database? It's what i've always done, but i've been wondering is all.
speaking of migrations.
that's a bloody minefield if ever i came across one
 
It's just easier to work with one DB than with N DBs
Are we talking mysql or postgresql or something else?
 
@nikita2206 i guess it's one of those it might appear great in theory but in practice you'd realise pretty quickly that you'd need data from another DB and table.
migration: mysql
we use Doctrine which is great, but i worry about how it does builds the DB.
 
Yeah, mysql is a pain by itself...
 
seems ok with small DB sizes but then it quickly becomes difficult
is there a better storage engine i should use?
I'm tried being webscale, no dice ;)
i've*
 
postgresql
 
12:12 PM
2 mins ago, by Andy
I'm tried being webscale, no dice ;)
 
i'm in the process of googling it, but what are the main advantages over mysql?
 
@Ocramius are green ZF elephpants still a thing and how do I get my hands on one?
hint hint
 
@Andy functionality, but this has nothing to do with you primary issue
40GB database is not all that impressive
I would see no point in splitting it
 
@tereško ref: 40Gb i know, i'm worried about when it gets to 100. So wanna see what i can do now
 
you should worry only when it's hitting TB line .. and only regarding aspects of backups
 
12:14 PM
100 is not a lot as well man
 
Good!
it is when you're a small company of 3 :)
so, for now, mysql should still what we use? Or rather, why would i change?
 
if your DB is growing uncontrollably, you might try to explain us "why"
 
postgresql, for example, can handle transactional DDL
 
hahaha
it's not thankfully
growing at an alarming rate.
it's more a thought experiement, would breaking it up improve the overall performance and reduce some potential issues. (of which i dont know)
 
@Andy migration of RDBMS is a really really large change
 
12:17 PM
@tereško dont i know it! I've done it before from MySQL to Web Scale, but argh, that was annoying. (I appreciate that Mongo aint an RDBMS, but still)
any large scale migraation is crap
 
@Andy Because if and when you reach the point where mysql just doesn't cut it, migration is a real pain.
My company's at that point now.
And I can tell you, it's a very real, very painful pain.
 
plus, as the owner of the company, it's the kind of problem i'd like to have, shows we're growing. Hopefully if it comes to that i can pay someone to manage it all :)
 
what do you mean with "web scale" ?
 
yeah wtf is web scale
 
have you not heard of this?
 
12:18 PM
ah youre joking
 
yep, hence the ;)
 
please stop
 
joking? :( poor me
 
sarcasm is not compatible with TCP/IP
 
@Andy I have been using Mongo for almost a year now, and I've encountered no problems whatsoever.
 
12:19 PM
true!
 
Everything is intuitive and easy, I spotted no performance problems and no real pains
I don't get where all this hate comes from.
 
@SecondRikudo the concern we have is that we were worried it might not save every time plus duplicate data
 
@Andy Storage is cheap
Time is not
 
@SecondRikudo never has a truer thing been said ... well maybe sometimes.
 
In 2015, I'd rather have a 10GB database that can fetch any full record in under 50ms, and not a 100MB database that takes 2 seconds to respond.
It of course depends on what you are doing
But for instance, fetching a page, all of the comments and all of the users is an O(n^3) with a relational database and joins
With mongo it's O(1)
Mongo loses to relational with O(1) vs O(1) (i.e. if all the data you ever need for the problem is found in one table)
But wins when things get complicated.
 
12:22 PM
so riddle me this, i run a price comparison whitelabel company -- fun right? I knows right? Anyway, we're looking at better ways of storing the product data so that we can scale it up and build our own search. We were thinking something like mongo as a product is a document per se. But each prodyuct then has a category. So already if you have 2 prods within the same cat you've got tyhat cat stored twice. So updating the category is difficult.
 
O_O
 
unless we've completely misunderstood something?
 
@Andy Have one category document that has many product documents nested inside.
Unless a product can have multiple categories?
 
my personal thought is this data would be best stored in a GraphDB of sorts, as when you boil it doen a product is just a collection of nodes. Name, specs, images, offers etc...
 
@SecondRikudo or unless he has a lot of products
 
12:24 PM
@SecondRikudo yes, it can have both plus we have at least 40M products. that's before we build our scraper and start looking at CPA product feeds.
so it just seems like a lot of duplicate data that's difficult to mantain, whereas a RDBMS is easier for this, normalizing and that. But then search is tough.
 
@Andy In that case, you can still go with the normalized approach
 
which is why it keeps leading me back to Graph ....
 
It's just that instead of a JOIN you have two queries
 
GraphDBs are beautiful but no one knows how to work with them... (me included) if you look at the gremlin language it's so powerful...
 
i think a doc based approach could work, but probably better with something like Elastic and having something indexing the tables.
 
12:26 PM
[{ _id: ..., categories: [list, of, ids], otherInfo: ...}, {...}]
 
then i can just index as and when i need and also rebuild should i want to change what i want to search on.
plus it's rapido -- which is always god.
@nikita2206 I've noticed that, i've been thinking that Graph would be perfect for this and have done for years. But i dont know anyone technical enough to even try and MVP it and my JAVA aint up to scratch anymore
 
@SecondRikudo so how do you filter by category?
 
last time i looked was with Neo4j
 
@nikita2206 categories: categoryId
Mongo searches collections by default.
 
ok @SecondRikudo how about if i'm looking for "Black Televisions that are with in the Plasma TV Category have offers from Merchant X,Y,Z but NOT A and is within 200 and 300 £"
i've found that sort of query difficult to build with something like mongo unless you havea huge document for the product
 
12:29 PM
@Andy All of the above is possible.
 
but merchant offers change daily
i know, but it's the storage
 
@Andy Of course the product needs to be a huge document
 
then what if i want to add another element to the doc to say, oh fuck what about this.
 
How can you search by criteria that don't exist?
 
i know! that's my point. At least with Graph the above is already built in
(as i understand it)
and is scalable, vs. something document based that would need constant revision (what if a provider suddenly adds condition of the product, but others dont?) that sort of thing.
 
12:31 PM
I'm not familiar with Graph.
@Andy Then you add it to the appropriate documents...
There's no schema
The only requirements from a document are the indexed fields
 
well the way i understand it is that all the above criteria, Colour: Black, Category: Plasma TV are all relationships between products
and graph searching is based on relationships and how many products have the ones you're after etc...
 
{_id: abc, name: Madara} and {_id: cba, name: Hashirama, age: 122} are both valid documents able to live on the same collection.
 
so it's "simple" to add more and more indexes as your data grows without needing to worry about much else.
yes but the Madara doc doesnt have age, when it should. But i can see what you mean. the search will ignore stuff that doesnt have the column.
 
So one product may have "Color" and the other may have "Total calories"
And they are both completely valid
And if you try to search for things with the color black, it will not throw an error for all the documents that don't have color, it will simply ignore them
Which is the expected behavior.
 
i can see the benefit, but i'm not sold that it's a viable storage engine for a product database
be good for something like Imgur.com or something like that
 
12:34 PM
@Andy What are you missing?
You can place whatever crazy criteria you want, you can read all matching entries with one read.
The cost is storage space, but storage is cheap
 
i'm just not convinced.
for this product at least.
 
vOv I tried my best
 
i'm trying to put my thoughts down
;)
you did. You've given me alot to think about for future projects though
 
So far you haven't presented a problem that Mongo can't solve elegantly.
 
indeed :)
 
12:36 PM
There are two places where I know you don't want mongo
1. When money or lives are involved
2. When all the data you need can be stuffed in one table (i.e. absolutely no relationships with other entities)
1. is because Mongo doesn't have transactions. If you want to get an absolute YES or NO on whether a query was fully executed, mongo is not for you.
2. is because in that case an RDBM is much faster.
 
:)
 
@SecondRikudo where did you learn about mongo? A site or a great book is appreciated.
 
^ ACID
 
right o folks, it's been great chatting. GTG
 
@HamZa Their own docs are pretty good.
 
12:38 PM
I once just saw a video "from mysql to mongo" and now I forgot everything :)
 
I learned most of it from there, the rest from messing around
 
Makes sense.
 
3. A place where data analysis is involved
4. A place where you don't want your data to be in inconsistent state (which happens because of denormalization and/or lack of transactions)
It's really not tied to money, there are lots of other places where you don't want to introduce inconsistency...
 

« first day (1564 days earlier)      last day (3381 days later) »