greptilian logo

IRC log for #javaee, 2013-09-11

Please see http://irclog.greptilian.com/javaee for which days have been logged.

| Channels | #javaee index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

All times shown according to UTC.

Time S Nick Message
00:04 sfisque packing up.  code strong!
00:46 sfisque joined ##javaee
00:46 sfisque1 joined ##javaee
00:54 * Quest waves
01:00 * sfisque waves back
01:12 * tjsnell yawns
03:13 [[thufir]] joined ##javaee
03:50 SoniEx joined ##javaee
04:35 Jazz joined ##javaee
04:35 Jazz Hey guys, I have a java situation... I have double data type that needs to be passed to a class that only takes int types.. what do i do? :s
04:36 grug really...
04:37 Jazz I cant change either type
04:37 Jazz :(
04:37 Jazz I'm new to java
04:37 grug well then you need to cast it
04:37 grug and you will lose precision
04:37 Jazz ok..
04:38 grug also don't forget, there is ##java - this is a (shitty) channel that claims to specialise in java ee
04:38 Jazz so I have.. double x = 1; and Sumclass(x) but only takes int..
04:38 grug i've already told you the answer
04:38 grug if you can't change the types that Sumclass takes
04:38 grug then you need to cast
04:39 grug why can't you change the types?
04:39 tangatools joined ##javaee
04:40 Jazz the class I have to pass the parameters is written to take only int, I'm not allowed to change that
04:40 grug why aren't you allowed to change it?
04:40 Jazz but I have numbers such as .. 4.5 so i have to use double
04:40 Jazz because its part of the java.awt.Rectangle class
04:45 grug you can't use something like java.awt.geom.Rectangle2D.Double instead?
04:45 Jazz let me look into it
04:46 Jazz wow, yeah thanks. :p
04:46 grug no worries
04:47 grug just to make sure you understand - you get why casting your double values to ints would be a bad idea right?
04:47 grug you understand the consequences of precision loss?
04:48 Jazz yes i do
04:48 grug cool
04:48 Jazz I didnt know that this existed, the Rectangle2D.Double
04:48 grug it's linked on the java.awt.Rectangle javadoc page
04:49 grug http://docs.oracle.com/javase/6/docs/api/java/awt/Rectangle.html
04:49 grug :)
04:50 Jazz I was just googling to check if what I did was the most efficient way or not.. finding the cords and area of rectangle intercepts
04:51 Jazz my first one looked like it was wayyyyy to many lines of code for something so simple
04:51 Jazz lol
04:53 grug another protip: join ##java
04:55 Jazz i dont feel like identifying myself
04:55 Jazz lulz
04:59 grug that seems odd...
05:01 Jazz lol :p im lazy
05:01 grug it takes like 2 seconds to auth
05:03 Jazz eh
05:03 Jazz they add up..
05:03 Jazz lul
05:03 Jazz z
05:04 grug what...
05:04 sfisque you know rectangle supports both double and int , right?
05:04 znurgl joined ##javaee
05:05 grug sfisque: the constructor doesn't though
05:05 sfisque who cares.
05:05 Jazz Rectangle from awt.Rectangle only takes an int, well you could pass a double but it aint gonna output so nice
05:05 sfisque 0,0
05:05 sfisque and then use the mutators to change to the double values y ou want
05:05 Jazz ok have fun doing that with numbers like... 73.168134
05:06 Jazz lulz
05:06 grug Jazz: i dont think you understand what he's saying
05:06 Jazz I dont?
05:06 grug nope.
05:06 Jazz Okeh:(
05:06 grug how about having a read through the javadoc - in this case, both sfisque is right, and i am right
05:06 grug you could use either (technically)
05:07 grug anyways - meeting time
05:07 Jazz ok thanks for the help
05:08 grug i don't quite think you've fully understood the help you've received :P (read the javadocs so you understand sfisque's idea)
05:08 Jazz I just started reading the rect stuff about 2-3 min ago :p
05:09 sfisque let me check the jdocs, i'd be surprised if Rectangle doesnt have a default constructor even.
05:09 Jazz wait..
05:09 Quest i am starting many threads in  a for loop with instantiating a runnable class but it seems I am getting same result from db (I am geting some string from db, then changing it) but with each thread, i get same string (despite each thread changed it.) . using jdbc for postgresql     what might be the usual issues ?
05:09 sfisque yah it totally has a default constructor
05:09 grug sfisque: it doesn't
05:09 grug well it does
05:10 sfisque it does.  has to , to comply with javabeans spec
05:10 grug yep
05:10 sfisque so you can do things like use a visual layout tool
05:10 Jazz boolean values in java, a false boolean type and trying to use a comparator, what is that like for java?
05:10 grug anyways have to go to a meeting now
05:10 Jazz ok bye
05:10 Jazz good luck
05:10 sfisque be well
05:11 Jazz if boolean x is false.. is if(x == 0) correct to compare?
05:11 Quest no
05:11 sfisque you have to choose an ordering.  is false > true or true true > false
05:11 Quest booleans have either ture or false value
05:11 sfisque comparator assumes an ordering
05:12 Jazz so (x == "false") ?
05:12 sfisque -1 less than, 0 equal, 1 greater than
05:12 Quest Boolean might have a null value if iam coorect as its a wrapper
05:12 Quest an object
05:12 Quest Jazz,  without ""
05:12 sfisque aye but comparators return an int
05:12 sfisque the contract is −1, 0, 1 depending on ordering
05:12 Quest if (x) { do some thing }
05:12 Jazz really? then why the.. i hate eclipse.. never lean an new language in an IDE.. but course requirements.. bleh
05:13 Quest if (x) { do some thing }       is same as       if ( x is true ) {do something}
05:13 sfisque Method Summary
05:13 sfisque int
05:13 sfisque http://docs.oracle.com/javase/6/docs/api/java/lang/Comparable.html#compareTo%28T%29(http://docs.oracle.com/javase/6/docs/api/java/lang/Comparable.html o)
05:13 sfisque Compares this object with the specified object for order.
05:13 sfisque
05:13 Jazz would if(!=x) work?
05:13 sfisque from java.lang.Comparable
05:14 sfisque X.compareTo( Y ) { if ( x < y ) { return −1 } else if ( x > y ) { return 1 } else return 0; }
05:14 sfisque or reversed depending on your ordering
05:14 sfisque zero == equal
05:15 Jazz ok, thanks
05:15 sfisque you also need to do "type" checking and throw a ClassCastExcpetion if the submitted Y is not an equivalent type (Class.isAssignable())
05:16 Jazz ok
05:17 sfisque http://docs.oracle.com/javase/6/docs/api/java/lang/Comparable.html
05:17 sfisque read the header.  it details the contract
05:17 Jazz ok will do thanks
05:17 sfisque npnp
05:17 sfisque and on that note, i'm heading to bead.
05:17 sfisque *** bed
05:17 sfisque code strong!
05:18 Jazz haha ok, goodnight
05:19 Quest any one free to see my code (long code but just glance at what might be wrong)
05:22 Jazz sure
05:22 Jazz i might be new but i have a psuedo code background
05:22 Jazz perl & js
05:23 Jazz I know sometimes code plays optical illusions on you if you look at it for to long
05:24 manulite joined ##javaee
06:31 neuro_sys Do you think if it's possible for a jax-rs method to take both form params as well as a complex type from request body as xml?
06:50 SoniEx joined ##javaee
07:50 SoniEx joined ##javaee
08:05 [[thufir]] joined ##javaee
11:47 Quest joined ##javaee
12:08 [[thufir]] joined ##javaee
12:09 neuromancer joined ##javaee
12:10 neuromancer boo!
12:10 acuzio where is Quest
12:11 neuromancer he is looking for the holy grail
12:12 neuromancer but didn't know the air speed velocity of a ladden swallow, and got into trouble
12:21 pdurbin heh. perl and js as pseudocode :)
12:23 acuzio neuromancer: laden
12:26 neuromancer laden it is
13:04 neuro_sys joined ##javaee
13:09 neuro_sys joined ##javaee
13:22 kobain joined ##javaee
13:24 Naros joined ##javaee
13:24 Guest59131 joined ##javaee
13:25 Naros sfisque: got a sec?
13:26 acuzio is Quest here ?
13:26 acuzio I wanted to ask him a question ?
13:26 Naros I don't see him acuzio but perhaps the rest of us could help?
13:26 acuzio no no its only him
13:27 Naros Alright then.
13:27 acuzio Sorry Naros nothing against you but he is very aware of the problem
13:28 Naros No offense taken, just trying to help was all.
13:28 acuzio do you work with Quest ?
13:29 Naros No I do not.
13:29 Naros I just know of him from here and help him when he has questions.
13:33 * tjsnell yawns
13:36 acuzio Naros: Thanks
14:09 Guest59131 Naros, free?
14:24 Quest any one good at concurrency?
14:27 pdurbin Quest: this guy: http://www.javapubhouse.com
14:28 Quest lol
14:29 tjsnell got a copy of JCIP?
14:31 pdurbin crimsonfubot: lucky java JCIP
14:31 crimsonfubot pdurbin: http://jetbrains.dzone.com/tips/concurrency-hot-try-jcip
14:31 pdurbin ah. Java Concurrency in Practice: Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, Doug Lea: 9780321349606: Amazon.com: Books - http://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz/dp/0321349601
14:31 tjsnell if you're doing java concurrency it's pretty much a must have
14:33 Quest ya. I just made up an app and all threads proccess the same link (despite when one link is processed its marked true. and every thread tries to process a false marked link)
14:33 Quest here is some thing fishy
14:34 tjsnell concurrency is very non trivial
14:34 tjsnell don't skip the in depth learning
14:35 pdurbin it's why Go was invented ;)
14:35 Quest tjsnell,  hm tried volatiles.. synchronized {} blocks.     thread start delays. no use
14:36 tjsnell yes, ignore my advice. Continue to have issues
14:36 * tjsnell shrugs
14:37 Quest i thought that was what you meant by depth..
14:38 tjsnell no I mean buy jcip, read/study/apply
14:38 Quest whats jcip?
14:39 tjsnell read the scroll
14:39 Quest hm
14:39 Quest oh.
14:45 sfisque oh well, i missed naros
14:45 Naros sfisque: im here.
14:46 Naros sorry just buried in this SQL issue of mine.
14:46 sfisque oh, what's up.  you had a ? earlier when i was afk?
14:46 sfisque and pdurbin, nice link.  i've had that book for a LONG time (Lea == concurrancy bible)
14:46 Naros yah i have a sql update im trying to do where I need to apply an aggregate to several columns in a table.
14:46 Quest I just made up an app and all threads proccess the same link (despite when one link is processed its marked true. and every thread tries to process a false marked link)    http://stackoverflow.com/questions/18743828/multiple-threads-using-same-value-fetched-by-database-dao-class-method
14:47 Naros cant determine how to run an update with an aggregate without using subqueries which is horrible for performance.
14:47 sfisque hrm, i've not done that in jpa yet.  always did single collumn when doing batch updates
14:47 Naros UPDATE table set field1 = (subquery), field2 = (subquery) where conditions...
14:47 sfisque right
14:47 sfisque i've done it a billions times in raw sql, never in jpa
14:47 Naros i can do a select aggr1, aggr2, aggr3 from tables left outer joins etc where conditions
14:48 Naros but to apply that select to an update is where it doesn't seem to jive other than iterate the cursor
14:48 Naros oh im talking raw sql
14:48 Naros this is part of a bulk data update phase that happens during a data load outside of jpa
14:50 Naros hm maybe use a WITH clause followed by update might work
14:51 sfisque are you doing updates to columns in different tables with the same update?
14:52 sfisque oh, i see, you're doing sum()'s and count()'s and injecting those into the collumns
14:52 Naros Nope, all in the same table
14:53 sfisque erm… that's a level of sql that is beyond me.
14:53 Naros I think the syntax WITH x AS ( select aggregate query here ) UPDATE table SET fieldA = x.fieldA, fieldB = x.fieldB FROM x WHERE join table with x group columns
14:53 sfisque things like that, i go to "the dba dudes" across the office :-)
14:53 Naros lol
14:55 sfisque long ago, i learned that i cannot do everything, so i came to understand when to delegate.  as an old man once said, you gotta know when to hold'em and know when to fold'em
14:56 pdurbin know when to walk away
14:58 sfisque :-)
15:01 sl33k1 joined ##javaee
15:02 sl33k1 i have a javaee question...i need to setup spring + jersey in a web app....but i dont see how to get my spring beans injected into my jersey rest services
15:19 Naros sfisque: updated over 25 million records in under 60 seconds with that syntax
15:19 Naros literally was taking hours using the UPDATE subquery aggregate DDL statement
15:20 sfisque very nice naros
15:21 sfisque sl33k1 - why do you need spring if you're in an EE container.  you should be able to annotate them as @Stateless or @Named and inject them that way
15:21 Naros unless he really wants to use spring
15:22 Naros in which cause you'd use @Inject or @Autowired iirc
15:22 Naros *case
15:22 sfisque aye.  but i'm assuming he's done that and the injection is not working
15:22 Naros but I believe your jersey rest services need to be allocated by spring too as it's object factory
15:22 sl33k1 sfisque: thats not the question. 2) i am not in  an ee container
15:23 Naros iirc, struts didn't play nice with spring for injection until we allowed spring to create our actions
15:23 Naros but dont quote me on that - was like 10 years ago :P
15:23 sfisque ah, so you're in a servlet container using spring and jersey?
15:25 sfisque i'm guessing you need some spring factory to instantiate/manage the jersey endpoints.  docs?
15:26 Naros you need to use com.sun.jersey.spi.spring.con​tainer.servlet.SpringServlet inside web.xml
15:26 Naros http://stackoverflow.com/questions/6672701/jersey-and-spring-integration-bean-injections-are-null-at-runtime
15:27 Naros chances are jersey is creating the objects rather than delegating to spring as I indicated.
15:28 sl33k1 Naros: yeah, saw that too...wonder why their example here: https://github.com/jersey/jersey/blob/2.2/examples/helloworld-spring-webapp/src/main/webapp/WEB-INF/web.xml doesnt even show this
15:31 Naros What container are you running under?
15:31 Naros that example implies glassfish
15:33 Naros I would suspect things might behave slightly different under Glassfish which is a JavaEE container rather than say Tomcat which isn't.
15:34 sl33k1 Tomcat.
15:34 Naros aye, then that web.xml won't help you in that example.
15:34 sl33k1 Naros: thanks . let me try that
15:34 Naros you need to reference the one from stackoverflow.com
15:35 sl33k1 yeah, i seen that but was looking for a reference in Jersey Docs. their chapter linked to that github which didnt fit.
15:36 Naros Aye, this relates to sfisque's question about whether you're using a JavaEE container or not since a JavaEE container != Servlet container.
15:40 sl33k1 I already answered this. I am using a Servlet Container.
16:01 sl33k joined ##javaee
16:42 kobain_ joined ##javaee
16:42 kobain_ joined ##javaee
16:45 Quest any one used row locking or faced a problem of Select statements not seeing the Update effect on a row in concurent threads?
16:47 whartung Look up SQL Isolation Levels Quest
16:48 sfisque if your provider has a L2 cache, you might be experiencing "dirty reads"
16:48 Quest whartung,  ya. i have tried row locks for selects.    (update does locks auto in PG)
16:48 Quest whartung,  sfisque  (not using hibernate).   http://stackoverflow.com/questions/18743828/multiple-threads-using-same-value-fetched-by-database-dao-class-method#18744696     start with  last comment of answer
16:49 whartung how is this not a SQL Isolation issue?
16:50 regedit joined ##javaee
16:50 regedit left ##javaee
16:51 Quest whartung,  what do you mean? can you elaborate
16:52 whartung SQL Isolation Levels -- they determine what you can and cannot see from a database when multiple transactions are taking place simultanesouly
16:53 Quest hm whartung  that is where row / table locks come in?
16:54 Quest whartung,  so yes.  i have tried all these. no use. http://www.postgresql.org/docs/current/static/explicit-locking.html
16:55 whartung it controls what changes to the DB you see that other are doing
16:55 whartung it doesn't affect locking at all
16:56 Quest hm. how can i control that behaviour in pg?
16:56 Quest whartung,  or do i just need to set me queries right
16:56 whartung http://www.postgresql.org/docs/9.1/static/transaction-iso.html
17:01 Quest it says. UPDATE, and SELECT FOR SHARE commands behave the same as SELECT in terms of searching for target rows: they will only find target rows that were committed as of the transaction start time. However, such a target row might have already been updated (or deleted or locked) by another concurrent transaction by the time it is found. In this case, the repeatable read transaction will wait for the first updating transaction to commit or roll back (if i
17:01 Quest t is still in progress).
17:02 Quest looks the solution but thats what i did already.   2.  those are two queries that run on different occasions         to mark : update links set processed = ? where href = ? and to get : SELECT href from links where processed = false limit 1
17:02 tjsnell holy paste!
17:03 Quest whartung,  the real problem is. The problem with update - its changes are not seen to other crawlers until a transaction is committed
17:03 Quest well even minutes after the update happens.    the selects see them unchanged
17:06 Naros Quest: if an update happens but hasn't been committed, that's normal behavior.
17:06 Naros if an update happens and has been committed but another session still reads dirty data, then i'd ponder if you're using some cache mechanism
17:07 Naros ehcache for example will report the stale data for up to it's time-to-live before the database is requeried.
17:07 Quest Naros,  i dont understand why even after the updat happens. other selects see it as unchanged
17:07 Quest Naros,  how to be sure i am not using cache?
17:07 Quest PG by default dont  i think
17:07 Naros if you start a transaction in SQL and run an update on a record but never commit the transaction or roll it back, those changes are only for your session to see, not any others.
17:08 Naros once a commit happens, the new changes can be seen by a new SELECT in another session.
17:08 Naros i speak of cache from an entity point of view using hibernate for example.
17:08 Quest run an update . and dont commit ? what do you mean?         query "update query bla bla". execute(); in jdbc comits it
17:09 Naros or perhaps where you have an in-memory datastore on the app tier side to avoid repetitive DB queries in short windows
17:09 Naros Only if auto commit is enabled :P
17:09 Naros JDBC can have auto commit disabled
17:09 Quest Naros,  not using hibernate
17:09 Quest ya. that is what i suspect. some cache
17:10 Quest Naros,  what do you mean by commit in ref to jdbc?
17:10 Quest the syntax i mean in a prepared statement
17:10 Naros http://docs.oracle.com/javase/tutorial/jdbc/basics/transactions.html#disable_auto_commit
17:10 Naros you can tell JDBC not to commit each DDL statement
17:11 Naros this is usually done when multiple DDL statements are part of a larger transaction
17:11 Quest When a connection is created, it is in auto-commit mode. This means that each individual SQL statement is treated as a transaction and is automatically committed right after it is executed.
17:11 Quest thats what i needed i gues
17:11 Naros Just call conn.commit() after your update and see if you then can see the changes
17:12 Quest hm.. ok
17:12 Quest let me seeee...
17:12 Naros A lot of this is also somewhat reliant on the driver you're using too
17:12 Naros e.g. JDBC can scroll result sets from modern DBs but older MySQL drivers would fetch the entire result set to the java side
17:13 Naros even when u tried to scroll and step it 1 by 1
17:13 Quest Naros,  con.commit() after execute or before?
17:13 Naros after
17:13 Naros typical SQL is
17:13 Naros BEGIN TRANSACTION
17:13 Naros do stuff
17:13 Naros COMMIT TRANSACTION
17:13 Naros somewhere in there after the BEGIN you have some catch code where you call rollback on failure.
17:14 Naros If you look at http://docs.oracle.com/javase/tutorial/jdbc/basics/transactions.html#commit_transactions you'll see a decent example
17:14 Naros they run two updates in a transaction and commit upon success, rollback on failure.
17:15 Quest giving you a paste
17:16 Quest http://pastebin.com/azYDMkdU
17:22 Naros and you can't see the values after that runs?
17:22 Quest values?
17:23 Naros when u query the table, do you not see the changes?
17:23 Quest I have just re ran the app with that line added . con.comit()
17:25 Quest Naros,  same results
17:25 gelmernan joined ##javaee
17:26 Naros and you're sure the sql is running?
17:27 Quest let me add a sysout of sql query
17:30 Quest Naros,  by the way. org.postgresql.util.PSQLException: Cannot commit when autoCommit is enabled.
17:31 Naros yes, as i pointed out earlier its general practice to disable auto commit
17:31 Naros and manually issue commit/rollback as needed
17:34 Quest Naros,  each thread is working on same link. making it processed = true and then again the other thread is working on it (despite each thread fetches processed = false ones) . doing same again. http://pastebin.com/2S3301gD
17:36 Naros oh gawd
17:37 Naros There are a few ways you can go about this
17:38 Naros 1) get your result set of links and give each thread a subset of the master result set
17:38 Naros lets say you query and get 100 links
17:38 Naros and you have 10 threads, give thread[0] records 0-9, thread[1] records 10-19, etc...
17:39 Naros if you're going to have the individual threads querying state, you're going to have to develop some row locking mechanism
17:39 Naros SELECT with intent to UPDATE so that another thread can't select the same record
17:39 Quest Naros,  i have trieda ll in http://www.postgresql.org/docs/current/static/explicit-locking.html       for row locking
17:41 Naros i understand but the issue appears to be that each thread tries to update the same record
17:41 Naros so you update record 1 10 times, record 2 10 times, etc.
17:42 Quest ya.  and it tries that because it sees it as proccessed = false.
17:42 Quest despite. it had already been processed
17:42 Quest thats what i was trying to understand
17:43 Quest or put those dao methods in a synchronized block?
17:43 Naros What you probably have is some SELECT that is getting the link name right?
17:43 Quest Naros,  yes.
17:43 Naros Where do you have this select, in the thread?
17:43 Quest public boolean markLinkAsProcesed(String link){                        public String getNonProcessedLink(){
17:43 Quest to mark : update links set processed = ? where href = ? and to get : SELECT href from links where
17:44 Quest should i those dao methods in a synchronized block?
17:44 Naros Quest...
17:44 Naros Where are you doing SELECT href from LINKS?
17:44 tjsnell use a message system and one thread to handle the db that consumes from the message queue
17:45 Naros if you have 10 threads, each which fire SELECT href from LINKS and then tried to update the value, then all 10 threads are going to do the same thing
17:45 Quest just a bad copy paste. to mark : update links set processed = ? where href = ? and to get : SELECT href from links where processed = false limit 1
17:45 Naros you basically should do this 1 of 2 ways
17:45 Naros the first way is
17:45 Quest hm. ..
17:46 Naros main thread does SELECT and then breaks the SELECT resultset into buckets for each thread to process their share
17:46 Naros second way is
17:46 Naros don't use 10 threads, use one thread and pass them using the command pattern to this one background thread.  it uses a queue to pop, update until the queue is empty, then waits until more data arrives.
17:47 tjsnell message queues!
17:47 Quest the later is idea. or use executors . and worker threads. but.  I am just.... curiouse... why this is happening.    this might be a php  / javaee website with many users quering.
17:47 Quest ideal*
17:48 Naros let me give you a real world example
17:48 Naros 2 users in 2 sessions query up a user record.
17:48 Naros both users change something about the same user record.
17:48 Naros they both hit commit at precisely the same point in time
17:48 Naros what do you think should happen?
17:48 Quest ya. i got you point but why my code is not doing what its supposed to do
17:48 tjsnell because the design is fundamentally flawed
17:49 Quest tjsnell,  which part
17:49 Naros can you paste where you are doing the SELECT statement please?
17:49 Quest Naros,  ya. both will be rejected i gues
17:49 tjsnell the idea that you can even do it that way and not have massive issues
17:49 Quest SELECT href from links where processed = false limit 1
17:49 tjsnell use a queue
17:49 Naros Quest: Nope that's not the case.
17:49 tjsnell then it's an easy problem
17:49 Naros One would happen just fine, the other depends on your design.
17:49 Quest tjsnell,  no i mean. which part is trouble some.
17:49 tjsnell all of it
17:49 tjsnell it's a fundamentally bad approach
17:49 Naros tjsnell isn't wrong here, he's spot on
17:50 tjsnell I would reject fixing it if shown at work
17:50 Quest tjsnell,  even that. but this does not makes sense. =  each thread is working on same link. making it processed = true and then again the other thread is working on it (despite each thread fetches processed = false ones) . doing same again.
17:50 tjsnell I would only suggest a redesign
17:50 tjsnell of course it doesn't
17:50 tjsnell bad design leads to confusing hard to debug code
17:50 Quest tjsnell,  I agree with that.  its bad approach
17:50 Naros Quest: a SELECT and UPDATE are two different calls to the DB
17:50 Quest tjsnell,  v
17:50 Quest I am just.... curiouse... why this is happening.    this might be a php  / javaee website with many users quering.
17:50 Naros if you issue 10 SELECTs before UPDATE ever happens, you can get this case.
17:50 Quest Naros,  yes
17:50 tjsnell BECAUSE ITS BAD DESIGN
17:51 Naros hence your design is indeed bad.
17:51 tjsnell I won't wade into a fundamentally flawed design and try to debug it
17:51 tjsnell ever
17:51 Naros either serialize the SELECT and distribute it to a worker pool or just use a queue.
17:51 Quest hm
17:51 tjsnell ~naros++
17:51 Naros tjsnell: what does that mean? <grin>
17:52 Quest i see
17:52 tjsnell Naros:  different channel and bot :)
17:52 Naros We face this same design in game programming where we need to update many models in parallel fashion.
17:52 Quest Naros,  why in the world , even row locking didnt helped up?
17:53 Quest http://www.postgresql.org/docs/current/static/explicit-locking.html
17:53 Quest despite its a bad design
17:53 Naros the solution is issue a query, get the result set, distribute the result set to a pool of workers to parallelize only the UPDATE, not the initial fetch.
17:53 Quest got it
17:54 Naros Quest: to your last question.  can you please pastebin where you do the select?
17:54 Naros i want to see the code itself
17:54 Naros maybe that might help you see the error on your design better.
17:55 Quest ok. will refactor threads.  one single thread works perfect though. so i can issue a beta.
17:55 Naros and multiple threads can work just as well if parallelism is necessary
17:56 Naros it's just you have to be careful how you approach threading
17:56 Naros certain things are great for parallelism, other things not so much unless you take extra caution
17:56 Quest Naros,  am just for discussion sake. how about i put the select dao calls in a synchronized block?
17:56 Quest so only one thread reads it at one time?
17:57 Naros if you have a method that just does a select and you synchronize that method call but have another method that does the update, it won't help unless they both synchronize on the same object
17:57 Naros Plus be weary of synchronized methods in a webapp
17:58 Naros What do u do if this webapp is in a cluster?
17:58 Naros Do you have the framework in place to handle synchronized methods across appserver boundaries in a cluster?
17:59 Naros If I take Quartz as an example, its a great jdbc store for background task execution in a cluster.
17:59 Quest hm
17:59 Naros it has a table it uses to row lock specific values
17:59 Quest Naros,  so i should sync both update and select methods?
18:00 Quest in same block?
18:00 Naros once the lock successfully is acquired, it then can safely operate on other tables.
18:00 Quest oh
18:00 Naros We do similar things where we maintain non-gapping sequence identifiers for documents.
18:00 Naros It comes at a performance cost ofc.
18:00 Quest hm
18:01 Quest Naros,  so i should sync both update and select methods in the same block. or just one of them
18:01 Naros imo, synchronized shouldn't be used here.
18:01 Naros either have the main thread SELECT href from LINKS WHERE ....
18:02 Naros get that resultset, split it up and give each thread work
18:02 Quest synchronized(something) { link = getNonProcessedLink(); markLinkAsProcesed(link); }
18:02 Naros or use a queue
18:02 Naros in the former, its a queue of sorts too
18:02 Naros just each thread has it's own queue
18:02 gelmernan joined ##javaee
18:03 Quest hm
18:05 Naros here try something like this
18:05 Naros 1 minute
18:07 Naros http://pastebin.com/ywKCt9qu
18:07 Quest Naros,  am. http://pastebin.ca/2448566
18:07 Naros now let me explain this
18:08 Naros this isn't workable code, just more notes than anything
18:08 Naros you select your hrefs in the main thread
18:08 Quest hm
18:08 Naros you construct some container for the link data.  This might just be adding a String to the queue in your simple case.
18:08 Naros myObject is anything
18:09 Naros create/start your thread subclass and pass it a reference to the queue
18:09 Naros inside the thread's run method you poll the queue
18:10 Naros http://pastebin.com/4puxdTa4
18:10 Naros inside the run method (threadMethod) you'll do a while and poll the queue
18:11 Quest hm
18:11 Naros You can get more sophisticated where the thread waits on a barrier, once notified threads wake up and process until no more data, then go back to sleep waiting on a barrier to notify it to work again
18:11 Quest ok
18:12 Naros but this kinda stuff is precisely why i said from the get go that threading isn't easy and most get it horribly wrong
18:12 Quest this didnt helped up. http://pastebin.ca/2448566
18:13 Quest hm
18:13 Quest ok.
18:13 Naros Using Hibernate/JPA you can issue a SELECT with a LOCK
18:13 Quest ya. hibernate makes it easy
18:13 Quest but Naros  http://pastebin.ca/2448566  should have worked
18:14 Naros Without access to the entire source repo to see what you're doing with these snippets, I can't say that is in fact true.
18:14 Quest hm
18:15 acuzio it should have worked
18:15 acuzio its synchronized
18:15 acuzio Quest: whats the actual problem ?
18:15 Naros where is url coming from?
18:15 Naros I see it referenced in that snippet, but it isn't in the synchronized block
18:16 Quest acuzio,   :) cant explain it the 59th time
18:16 Quest Naros,  from db
18:16 Quest Naros,  i just made it sync block
18:16 Quest in this http://pastebin.ca/2448566
18:21 Naros Quest: i honestly dont know what to tell you at this point.  people can show snippets of code all day, but without a workable or complete example of everything going on, there are just too many moving pieces for anyone to be able to adquately say one way or another.
18:22 Quest ya.
18:22 Quest hm let me see
18:22 acuzio Quest
18:23 acuzio why are you using synchronized ?
18:23 Naros I suspect its because he's doing threading.
18:24 Quest Naros,  line 88 .  its messy though . http://pastebin.com/ns5m7JgT
18:24 acuzio Naros: threading doesnt mean synchronized-  surely
18:25 Naros but i know he's doing threading :P
18:25 acuzio so what - that doesnt mean synchronized
18:26 Naros he's trying to prevent thread X from stepping on thread Y by processing the same links.
18:26 Quest the block means that the resources in it and code would only be acces by one thread at a time
18:26 Quest acuzio, ^
18:26 acuzio Quest: why not put it in the database ?
18:27 Quest it is the database issue
18:27 acuzio what is teh database issue ?
18:27 Quest hm . heres the details if you want to see acuzio  http://stackoverflow.com/questions/18743828/multiple-threads-using-same-value-fetched-by-database-dao-class-method#18744696
18:28 Naros Quest: question
18:28 acuzio Quest: is that your question ?
18:28 Quest kind of
18:29 Naros Quest: you create 10 Crawlers right?
18:29 Quest yup
18:29 Quest 10 crawler threads
18:29 Naros and each crawler is given a different url?
18:29 Naros or is it the same url?
18:29 Quest each gets a diff url by dao method
18:30 Naros ok, and each of those urls put different links in the DB?
18:30 Quest each gets a diff url by dao method by  public String getNonProcessedLink(){
18:30 Naros giva putLinksInDB(linkList) ?
18:30 Quest after that. it marks it true
18:30 Quest yes
18:30 whartung did you read the stuff about isolation Quest ?
18:30 Quest whartung,  yes
18:30 whartung how does that not apply here?
18:31 Quest I tried solutions . but none prooved succes
18:31 Quest whartung,  i really dont know what the problem is. really wanted to hack and larn
18:31 tjsnell Use a queue!
18:31 Quest learn
18:31 Quest tjsnell,  yes. agreed. but need to hack this
18:31 whartung queues solve everything
18:31 Quest whartung,  ya.
18:32 tjsnell why?
18:32 Naros they really do and you can have your crawler do a lot of what it does just in conjunction with a queue
18:32 acuzio tjsnell: you dont need a queue
18:32 Quest whartung,  by the way. dont have much memory for long ques  either
18:32 tjsnell a queue would work a lot easier and cleaner
18:32 tjsnell no one said anything about long queues
18:33 Quest whartung,  i dont even know why this problem is occuring (despite bad design . but i cant understand the badnes)
18:33 acuzio queues are not optimum - Quest is on the right track
18:34 Quest acuzio,  executors and pools for threads with workers is the right track. iam just hacking
18:35 acuzio i dont believe it
18:35 regedit joined ##javaee
18:35 regedit left ##javaee
18:35 * Naros goes back to working on fixing some table locking scenarios in TSQL.
18:36 acuzio we can just use Threads
18:36 acuzio and store in the database
18:36 Quest acuzio,  thats what i did
18:36 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:36 Quest Naros,  thanks!! again you dived alot
18:36 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:36 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:36 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:36 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer FUCKING MORONIC CHANNEL - PLEASE BAN ME
18:37 neuromancer left ##javaee
18:37 acuzio Quest: so why didnt that work ?
18:37 Quest acuzio,  details are in my SO post
18:38 acuzio Quest: did you ban neuromancer ?
18:38 Quest acuzio,  i did
18:39 tjsnell he seems upset
18:39 acuzio tjsnell: i bet its you
18:40 tjsnell nah
18:40 tjsnell I only use one id
18:40 acuzio i still blame you
18:41 Quest tjsnell,  upset? so am I with some channels. does getting upset matters to channels?
18:41 tjsnell apparently, you banned him for being upset
18:41 tjsnell I'm just making observations, not judgements
18:43 Quest i banned him for spaming
18:43 Quest but you didnt answer my question :)
18:43 pyBlob joined ##javaee
18:44 pyBlob left ##javaee
18:44 acuzio Quest: its a good question ., i am not sure why no one has answered it
18:44 Quest acuzio,  i see one answer to the post
18:44 acuzio I dont think its very clear - the answer
18:45 Quest it helped a bit to learn.
18:45 acuzio There are 10 threads each with a different starting time - they should then finish differently at access different URL's
18:45 acuzio this line is critical - boolean markedLinkAsProcessedBoolean = new BasicDAO().markLinkAsProcesed(url);
18:46 Quest ya.
18:46 acuzio markedLinkAsProcessedBoolean is a boolean that marks the link as processed
18:46 Quest the former may be incorrect though
18:46 Quest yes
18:48 acuzio yes
18:50 tjsnell Quest:  how can I answer that question.  It's non answerable.
18:50 tjsnell all channels are different
18:51 acuzio tjsnell: what question is non-answerable ? just because you dont know it does not mean its non-answerable.
18:52 tjsnell scroll is your friend
18:52 acuzio Do you mean Quest's question on threads ?
18:53 Quest acuzio,  no. he means answer to this "Quest> tjsnell,  upset? so am I with some channels. does getting upset matters to channels?"
18:53 tjsnell some channels yes, some no
18:53 tjsnell oddly enough
18:53 acuzio Quest: i told you  - he is a trouble maker -
18:53 acuzio tjsnell: did you shower today ?
18:53 tjsnell Quest:  sorry, it seemed a stupid question so I tend to ignore those
18:53 tjsnell acuzio:  thursdays
18:54 tjsnell weekend prep
18:54 Quest acuzio,  no teasing please
18:54 acuzio tjsnell: hence the stink
18:54 Quest acuzio,  no teasing please
18:54 acuzio Quest: what teasing -
18:54 tjsnell you hurt my feelins
18:54 acuzio Quest: its not teasing i am asking him a question
18:54 Quest acuzio,  ##javaee-offtopic then
18:55 acuzio there is an off-topic .- oh , tjsnell are you there ?
18:55 tjsnell nah, there's already java-talk so I don't see the point
18:55 Quest tjsnell is a senior , despite he has his own attitude. (liked by most or disliked)
18:56 acuzio Quest: despite he has his own attitude - what does that mean ?
18:56 acuzio Quest: are you using SQL Transactions ?
18:57 rooter_ joined ##javaee
18:58 Quest acuzio,  what do you mean ?
18:58 Quest acuzio,  iam using prepared statements. see basicDao.class
18:58 tjsnell some here don't have their own attitudes
18:58 acuzio Quest: So is there commit in it ?
18:59 Quest jdbc auto commits
18:59 Quest there is a pstatemetn.execut() though
19:00 acuzio Are you Masood Ahmed
19:01 Quest the user name is
19:01 acuzio what ?
19:02 acuzio so your real name is Quest and your username is Masood Ahmed ?
19:02 Quest the user name of the question poster is Masood Ahmad
19:02 acuzio Isnt that you ?
19:02 Quest does it matters :)
19:02 acuzio I remember this name from some place -
19:02 acuzio Are you from syncsys ?
19:02 rooter_ acuzio: he trolls often in #java, that's why ;)
19:02 Quest yes
19:02 rooter_ was kicked by Quest: rooter_
19:03 rooter_ joined ##javaee
19:03 acuzio why was he kicked ?
19:03 acuzio and banned
19:03 Quest acuzio,  dont you see ?  <rooter_> acuzio: he trolls often in #java, that's why ;)
19:03 acuzio that deserved a ban ?!
19:03 Quest for the moment. yes
19:05 Quest acuzio,  so . how long have you been in java world?
19:05 acuzio i am learning
19:05 Quest since?
19:06 acuzio seems like forever
19:07 Quest hm
19:07 tjsnell sometimes the rules here confuse me
19:07 Quest you didnt got used to them ?
19:08 acuzio tjsnell: whats confusing ?
19:09 acuzio Quest: he really is a troublemaker
19:13 acuzio Quest: so are you changing the code to fix autocommit
19:14 Quest commit is not needed. it does it by default at preparedStmt.execut()
19:14 Quest in jdbc
19:14 Quest http://docs.oracle.com/javase/tutorial/jdbc/basics/transactions.html#disable_auto_commit
19:14 acuzio so then it should work - are you sure you ran the program more than once
19:15 Quest its not.
19:16 acuzio you did not run it more than once
19:17 Quest 10 threads
19:17 acuzio Quest: you shld run it more than once may be it might work - threads you know
19:18 Quest single thread works fine
19:18 Quest acuzio,  no problem. for today. it enough on that topic. will continue tomorrow
19:18 acuzio so try running it multiple times - multiple threads you see
19:22 Quest :) see the code first.
20:23 SoniEx2 joined ##javaee
20:24 SoniEx2 joined ##javaee
20:35 Quest joined ##javaee
20:54 Fubar^ joined ##javaee
21:44 Quest joined ##javaee
21:51 SoniEx2 joined ##javaee
21:56 Quest acuzio, tjsnell,  whartung  Naros  for your interest and reference  (as you did got involved in my question). its solved by Fubar^  (thanks). answer at http://stackoverflow.com/questions/18743828/multiple-threads-using-same-value-fetched-by-database-dao-class-method
21:57 * Quest thanks all above for helping him explore things better and learn !
21:57 Naros Glad it's fixed.
21:57 whartung so what was the solution
21:57 Quest :)
21:57 Quest the solution is on the post
21:57 Quest as an answer
21:58 whartung what's the 5 words summary
21:58 Naros wasn't marking record processed before querying for new one it seems
21:58 Naros "It was the gap between marking a link processed true and fetching a new one and letting other threads get the same link while the current was working on it."
21:58 whartung since none of these are marked as accepted
21:58 whartung ah
21:58 Quest whartung,  a minut
21:59 Quest In SO . questioner cant aceept before 2 days
21:59 Naros which in all honesty was mentioned in our discussion but oh well.
21:59 Quest whartung,  5 line summary. It was the gap between marking a link processed true and fetching a new one and letting other threads get the same link while the current was working on it.
21:59 Quest Synchonized block helped further.
21:59 whartung ok
21:59 whartung maybe because you have a low rep
22:00 Quest Despite the comments and response about locks and databases by helpers in this post were also correct.
22:00 Quest whartung,  no. i guess. its a rule may be
22:01 whartung I've had answers accepted before 2 days unless it's a recent cahnge
22:01 Quest hm. then repo
22:02 sfisque keep in mind, you cannot use synchronized blocks in an EE container if you ever move the product from spring to ee.  you can deadlock the container and it's explicitly forbidden in the spec (though you CAN do it, but if you break it, you get to keep both halves :-)
22:02 Quest sfisque,  really?
22:02 whartung rily
22:02 sfisque chapter 24 iirc
22:03 sfisque in the list of "verbotten" things, like file i/o, spawning threads, etc.   NO synchronization primitives
22:03 Quest why in the world ee and servlet containers dont support sync blocks?
22:03 Naros There are constructs in the java.util.concurrent package specifically for doing similar things like synchronized but without synchronized.
22:03 whartung because they don't need to, they have other mechanisms
22:03 sfisque because you're in a pooled multithreaded env
22:03 Quest Naros,  for e.g?
22:03 whartung and it's not servlet containers, its jee containers -- thems different
22:03 sfisque if you're using synchronzied in an ejb, you're doing something wrong
22:03 Quest whartung,  oh. like?
22:04 Quest sfisque,  oh
22:04 Naros glassfish, weblogic, boss
22:04 Naros *jboss
22:04 Naros those are ee containers
22:04 whartung like MDBs, like async session beans,
22:04 Quest so the reason not to use sync blocks is that in ee we are already in pooled multithreaded environment
22:04 Quest ?
22:05 sfisque consider if you're in a SLSB, those are pooled and multi-threaded.  two instances can very easily get into the elevated priv deadlock
22:05 sfisque and if you're in a SFSB, you're holding state, and if any part of that state gets syncrhonized in another thread, your sfsb is hosed
22:05 whartung all that said, I use locks and blocking queues and other such things all the time, all of those use the underlying .wait mechanisms within java
22:06 sfisque aye outside the container (i.e. pojo, servlet, jca adapter, etc.) you're find and dandy
22:06 Quest hm
22:06 sfisque once you enter a sb transaction, you gotta tread lightly
22:18 [[thufir]] joined ##javaee
22:33 [[thufir]] joined ##javaee
23:23 [[thufir]] joined ##javaee

| Channels | #javaee index | Today | | Search | Google Search | Plain-Text | plain, newest first | summary

Please see http://irclog.greptilian.com/javaee for which days have been logged.