| Time |
S |
Nick |
Message |
| 01:15 |
|
|
sfisque joined ##javaee |
| 01:42 |
|
|
sfisque1 joined ##javaee |
| 04:23 |
|
|
Quest joined ##javaee |
| 04:47 |
|
|
dangertools joined ##javaee |
| 04:50 |
|
|
tangatools joined ##javaee |
| 04:50 |
|
|
tangatools joined ##javaee |
| 05:08 |
|
|
scripty joined ##javaee |
| 06:15 |
|
|
AlexCzar joined ##javaee |
| 06:37 |
|
|
Quest joined ##javaee |
| 06:37 |
|
Quest |
in idea, just imported my eclipse project that used struts2. its says HttpServletRequest cannot find symbol |
| 06:54 |
|
|
neuro_sys joined ##javaee |
| 08:02 |
|
|
tangatools joined ##javaee |
| 08:02 |
|
|
tangatools joined ##javaee |
| 08:19 |
|
Quest |
using idea and tomcat 6, jdk 6, all runs fine in other IDEs. getting this error. Application Server was not connected before run configuration stop, reason: Unable to ping server at localhost:1099. JavaeeServerInstanceImpl - Unable to ping server at localhost:1099 |
| 08:20 |
|
Fubar^ |
use one of the IDEs where it works then! :) |
| 08:21 |
|
Quest |
:) need idea only |
| 09:22 |
|
|
tjsnell left ##javaee |
| 09:38 |
|
|
neuro_sys joined ##javaee |
| 10:39 |
|
pdurbin |
Quest: did you figure it out? |
| 11:04 |
|
|
MegaMatt joined ##javaee |
| 11:14 |
|
Quest |
no |
| 11:14 |
|
Quest |
pdurbin, be right back |
| 11:39 |
|
* pdurbin |
is excited about a lot of the new features: Java Spotlight Episode 149: Geertjan Wielenga on NetBeans 7.4 @netbeans @geertjanw (The Java Spotlight Podcast) - https://blogs.oracle.com/javaspotlight/entry/java_spotlight_episode_149_geertjan |
| 11:40 |
|
pdurbin |
huh, Arun Gupta going from Oracle to Red Hat, they said at the end |
| 11:41 |
|
pdurbin |
Farewell from Oracle, Starting at Red Hat | Miles to go 2.0 … - http://blog.arungupta.me/2013/10/farewell-from-oracle/ |
| 12:06 |
|
|
cem_ joined ##javaee |
| 12:09 |
|
cem_ |
hi |
| 12:09 |
|
Fubar^ |
hello |
| 12:09 |
|
cem_ |
Fubar^: how to display chinese characters in eclipse |
| 12:10 |
|
Fubar^ |
where in eclipse do you want the chinese characters displayed? |
| 12:10 |
|
cem_ |
Fubar^: I'm getting like this å½¢å£°å— in editor |
| 12:11 |
|
Fubar^ |
maybe set preferences -> general -> workspace -> text file encoding |
| 12:14 |
|
cem_ |
text file encoding is utf-8 |
| 12:14 |
|
cem_ |
i need to change in os level |
| 12:14 |
|
cem_ |
? |
| 12:15 |
|
Fubar^ |
is utf-8 correct? |
| 12:16 |
|
cem_ |
means |
| 12:16 |
|
Fubar^ |
are your files encoded with utf-8? |
| 12:16 |
|
cem_ |
no clue |
| 12:17 |
|
Fubar^ |
you need to figure that out first |
| 12:18 |
|
cem_ |
text file encoding is utf-8 so it must encode in utf8 |
| 12:18 |
|
cem_ |
? |
| 12:19 |
|
Fubar^ |
the setting in eclipse just tells it what to expect |
| 12:24 |
|
cem_ |
how can i do that ? |
| 12:25 |
|
cem_ |
there is no property in OS to do that ? is there ? |
| 12:25 |
|
Fubar^ |
to do what? Figure out the correct encoding for your file? |
| 12:26 |
|
cem_ |
yes |
| 12:29 |
|
Fubar^ |
browsers are usually quite good at guessing encodings. Try opening your file in firefox and see how it looks |
| 12:29 |
|
Fubar^ |
If it looks good, check which encoding it uses |
| 12:32 |
|
cem_ |
its Utf-8 |
| 12:33 |
|
cem_ |
its looks awkward |
| 12:33 |
|
neuro_sys |
then it's not utf-8 |
| 12:35 |
|
cem_ |
neuro_sys: ?? |
| 12:35 |
|
cem_ |
Are you kidding me |
| 12:37 |
|
cem_ |
view > encoding > utf8 it shows |
| 12:38 |
|
Fubar^ |
but you're saying it doesn't look right |
| 12:38 |
|
neuro_sys |
It's impossible to perfectly guess the encoding. |
| 12:39 |
|
neuro_sys |
or well, there should be tools out there |
| 12:39 |
|
cem_ |
yes it looks not good 1 character is in chinese and other looks awkward |
| 12:39 |
|
Fubar^ |
then it's probably not utf-8 |
| 12:39 |
|
Fubar^ |
maybe try utf-16? |
| 12:40 |
|
cem_ |
its utf-8 i'll try utf-16 1 sec |
| 12:41 |
|
neuro_sys |
I'm curious, why do you think it's utf-8? |
| 12:44 |
|
cem_ |
Thanks Fubar^ its works fine i have been googling for a hour thanks |
| 12:47 |
|
|
sess joined ##javaee |
| 13:26 |
|
|
MegaMatt joined ##javaee |
| 13:29 |
|
|
Naros joined ##javaee |
| 14:11 |
|
sfisque1 |
if it's chinese, its probably utf-16. afaik, asian glyphs are mbyte |
| 14:46 |
|
Naros |
sfisque1: I figured out the issue btw. |
| 14:46 |
|
Naros |
seemed that Tomcat requires the Connector declaration in server.xml to have URIEncoding="utf-8" specified. |
| 14:46 |
|
Naros |
Otherwise it uses 8859-1 to decode GET information |
| 14:48 |
|
sfisque1 |
ah |
| 14:48 |
|
Naros |
seems weird but taht's what the docs claimed |
| 14:48 |
|
sfisque1 |
8859-1 is a windows encoding, iirc, no? |
| 14:49 |
|
sfisque1 |
javaeebot lucky 8859-1 encoding |
| 14:49 |
|
javaeebot |
sfisque1: http://en.wikipedia.org/wiki/ISO/IEC_8859-1 |
| 14:49 |
|
sfisque1 |
ay, iso eastern europe encoding |
| 14:49 |
|
Naros |
yah |
| 14:49 |
|
sfisque1 |
** ah |
| 14:50 |
|
pdurbin |
yar |
| 14:51 |
|
pdurbin |
urn:oid:1.3.6.1.4.1.5923.1.1.1.6 is a thing |
| 14:51 |
|
sfisque1 |
snmp |
| 14:52 |
|
sfisque1 |
from RFC 3986 - When a new URI scheme defines a component that represents textual |
| 14:52 |
|
sfisque1 |
data consisting of characters from the Universal Character Set [http://tools.ietf.org/html/rfc3986#ref-UCS], |
| 14:52 |
|
sfisque1 |
the data should first be encoded as octets according to the UTF-8 |
| 14:52 |
|
sfisque1 |
character encoding [http://tools.ietf.org/html/rfc3986#ref-STD63]; then only those octets that do not |
| 14:52 |
|
sfisque1 |
correspond to characters in the unreserved set should be percent- |
| 14:52 |
|
sfisque1 |
encoded. For example, the character A would be represented as "A", |
| 14:52 |
|
sfisque1 |
the character LATIN CAPITAL LETTER A WITH GRAVE would be represented |
| 14:52 |
|
sfisque1 |
as "%C3%80", and the character KATAKANA LETTER A would be represented |
| 14:52 |
|
sfisque1 |
as "%E3%82%A2". |
| 14:52 |
|
Naros |
o.O |
| 14:53 |
|
sfisque1 |
interesting results from nasa |
| 14:53 |
|
pdurbin |
sfisque1: yep yep, you'll find "pdurbin" at http://www.iana.org/assignments/enterprise-numbers/enterprise-numbers :) |
| 14:53 |
|
sfisque1 |
javaeebot lucky nasa says first space internet test beyond expectations |
| 14:53 |
|
javaeebot |
sfisque1: http://www.computerworld.com/s/article/9243426/NASA_says_first_space_Internet_test_beyond_expectations_ |
| 14:54 |
|
sfisque |
berklee college of music o.O |
| 14:54 |
|
pdurbin |
\o/ |
| 14:54 |
|
pdurbin |
awesome place |
| 14:55 |
|
sfisque |
party school? |
| 14:55 |
|
sfisque |
:P |
| 14:55 |
|
pdurbin |
heh. wasn't a student |
| 14:55 |
|
sfisque |
matters naught? |
| 14:55 |
|
* sfisque |
has partied with professors |
| 14:56 |
|
sfisque |
looks like the mit research for power over wireless is moving into cars |
| 14:57 |
|
sfisque |
javaeebot lucky auto industry cuts cord on electric car charging |
| 14:57 |
|
javaeebot |
sfisque: http://www.computerworld.com/s/article/9243346/ |
| 14:58 |
|
neuro_sys |
javaeebot: sup |
| 14:58 |
|
javaeebot |
neuro_sys: Error: "sup" is not a valid command. |
| 14:58 |
|
sfisque |
lol, did jeebot kick neuro? |
| 14:58 |
|
sfisque |
:P |
| 14:58 |
|
pdurbin |
heh |
| 14:59 |
|
sfisque |
so random question. anyone around here written or worked on a PAM module? |
| 15:01 |
|
acuzio |
sfisque: yes...... |
| 15:01 |
|
sfisque |
difficult? easy? insane? |
| 15:02 |
|
pdurbin |
sfisque: no, but these guys are and it's on our roadmap to integrate with them: https://www.irods.org/index.php/PAM_Authentication |
| 15:02 |
|
acuzio |
sfisque: PAM with NTLM - fucking insane |
| 15:02 |
|
sfisque |
i'm having thoughts about open source PAM tied to an open source "token generator". because, lets face it. RSA is not cheap :-/ |
| 15:03 |
|
acuzio |
sfisque: we have RSA - but we are finance - so thats ok ; i did PAM NTLM for an internal module ., it sucked |
| 15:04 |
|
sfisque |
with the NSA revelations, i'm starting to think that PKI with a key/pair is not enough |
| 15:04 |
|
pdurbin |
sfisque: https://github.com/fasrc/openauth ... didn't I link you to this already? |
| 15:04 |
|
sfisque |
aye. i havent had enough time to dig into the docs |
| 15:04 |
|
sfisque |
too many ideas, not enough time |
| 15:04 |
|
sfisque |
:-/ |
| 15:04 |
|
pdurbin |
sfisque: our previous discussion: http://irclog.greptilian.com/javaee/2013-08-08#i_17068 |
| 15:05 |
|
sfisque |
does oauth have a PAM module for linux? i'm concerned with securing my linux nodes with either hardware or software token generation |
| 15:05 |
|
|
zanatabr joined ##javaee |
| 15:09 |
|
pdurbin |
sfisque: someone in #crimsonfu might know: http://crimsonfu.github.io |
| 15:11 |
|
pdurbin |
or google might know :) |
| 15:11 |
|
pdurbin |
isn't oauth kind of tied to browsers, though? |
| 15:11 |
|
sfisque |
aye. this was my first stop after the idea filtered through my head. google was the next eventual step |
| 15:11 |
|
sfisque |
dunno, you linked it to me |
| 15:12 |
|
pdurbin |
:) |
| 15:12 |
|
sfisque |
i know its used in websites, but i am unaware of how flexible it is |
| 15:12 |
|
pdurbin |
my friends call me "linky" |
| 15:12 |
|
sfisque |
ROFL |
| 15:13 |
|
pdurbin |
whartung: I'm now able to have our app print "urn:oid:1.3.6.1.4.1.5923.1.1.1.6: myself testshib.org" using OIOSAML. pointed at the testshib Idp |
| 15:28 |
|
|
zanatabr left ##javaee |
| 15:57 |
|
whartung |
\o/ pd |
| 15:57 |
|
whartung |
pdurbin: |
| 15:57 |
|
pdurbin |
:) |
| 16:08 |
|
|
Quest joined ##javaee |
| 17:00 |
|
|
scripty joined ##javaee |
| 17:01 |
|
scripty |
ZZZ |
| 17:02 |
|
|
scripty joined ##javaee |
| 17:15 |
|
Quest |
Iam confused at something |
| 17:16 |
|
Quest |
if a class has a static variable. no matter how many times the class is instantiated. the static variable will remain same. it would be just one piece of variable in the world for that whole application. |
| 17:16 |
|
Quest |
are static varialbes in classes in a struts2 or spring application (where each class has a new instance every time) ran by tomcat, are shared as same like application scope? and are static varialbes in classes in a simple application made of bare servlets ran by tomcat, are shared as same like application scope? |
| 17:16 |
|
Quest |
so this http://pastie.org/8424567 will not make just ONE list of said variables for ALL requests/users/clients? |
| 17:19 |
|
* scripty |
giggles |
| 17:22 |
|
Quest |
ya.. i get confused. |
| 17:28 |
|
scripty |
javaeebot lucky tcm |
| 17:28 |
|
javaeebot |
scripty: http://www.tcm.com/ |
| 17:31 |
|
scripty |
javaeebot lucky taliban |
| 17:31 |
|
javaeebot |
scripty: http://en.wikipedia.org/wiki/Taliban |
| 17:31 |
|
pdurbin |
scripty: please stop |
| 17:32 |
|
scripty |
k |
| 17:34 |
|
scripty |
that site has lot of bugs wait ill inform to there admin |
| 17:40 |
|
cem_ |
hi quest san |
| 17:41 |
|
Quest |
hi |
| 17:42 |
|
cem_ |
it is a very big file |
| 17:42 |
|
Quest |
big file? |
| 17:43 |
|
cem_ |
yes quest it has 600+ lines of code |
| 17:43 |
|
cem_ |
very big |
| 17:43 |
|
Quest |
ya. choose your favorite portion :) |
| 17:43 |
|
cem_ |
nice Quest san |
| 17:45 |
|
cem_ |
what is Conncurenthashmap ? Quest san |
| 17:45 |
|
semiosis |
google concurrenthashmap |
| 17:45 |
|
semiosis |
javaeebot: lucky concurrenthashmap |
| 17:45 |
|
javaeebot |
semiosis: http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentHashMap.html |
| 17:46 |
|
cem_ |
is there anyway i can learn java faster i feel its so lengthy i am from C and i feel this language is very very lengthy |
| 17:48 |
|
Quest |
cem_, some kind of map that can be accessed / modified my multiple threads at the same time |
| 17:48 |
|
sess |
Quest: static variables are static regardless of spring or struts or whatnot |
| 17:49 |
|
Quest |
sess, good to see you. |
| 17:49 |
|
Quest |
sess, so we are good at just calling MyClass.staticMember a trillion times from any where at all. assuming that the value remains constant. so all threads/objects/instances of what ever class will see only ONE copy of that variable . there would be one copy in the memory also? |
| 17:51 |
|
sess |
the variable lies in memory |
| 17:52 |
|
sess |
and there is exactly one |
| 17:52 |
|
Quest |
hm |
| 17:52 |
|
sess |
you have to be careful if it's mutable |
| 17:52 |
|
Quest |
so its same as an application scoped variable |
| 17:52 |
|
Quest |
mutable? |
| 17:52 |
|
sess |
changeable |
| 17:52 |
|
Quest |
you mean can be broken in more objects? |
| 17:53 |
|
sess |
if it can be changed, there can be concurrency issues |
| 17:53 |
|
sess |
though i saw you use concurrent hash map |
| 17:53 |
|
Quest |
sess, oh ya. I would make it volatile then. or if its a map. make it concurrentmap |
| 17:53 |
|
Quest |
sess, thats not my code though |
| 17:53 |
|
sess |
oh |
| 17:53 |
|
Quest |
sess, so that is an ok code? |
| 17:54 |
|
Quest |
sess, or should it be better be using app scoped variables instead of those? |
| 17:54 |
|
sess |
im not going through 1000 lines |
| 17:54 |
|
Quest |
sess, just the first methods |
| 17:54 |
|
sess |
what are you trying to do |
| 17:54 |
|
sess |
a cache? |
| 17:54 |
|
Quest |
the intention is to load a large table into a Map in cache/ram (a variable). update it every 30 mins. and let requests not to be quering the database |
| 17:55 |
|
sess |
are you using struts or spring |
| 17:55 |
|
Quest |
struts |
| 17:55 |
|
Quest |
2 |
| 17:55 |
|
sess |
ok |
| 17:55 |
|
sess |
then id make a concurrent collection |
| 17:56 |
|
sess |
though you will have to lock it when it reloads |
| 17:56 |
|
sess |
or hmm |
| 17:56 |
|
sess |
if you load all data into a new map, then switch them, it could be fine |
| 17:56 |
|
Quest |
well how to lock it. good point. for the request that come between clearing it and reloading it |
| 17:57 |
|
sess |
if you prevent clients from getting the actual map |
| 17:57 |
|
Quest |
sess, oh greate. thats fine |
| 17:57 |
|
sess |
i.e make it private and proviate get to get value only |
| 17:57 |
|
Quest |
got it |
| 17:57 |
|
sess |
then you can just switch the map internally to a new one |
| 17:57 |
|
sess |
shouldnt even need to be concurrent |
| 17:57 |
|
sess |
since it will be in a proper state at all times |
| 17:57 |
|
Quest |
hm |
| 17:57 |
|
sess |
the old switcharoo |
| 17:58 |
|
sess |
granted you replace 100% of the data in the map |
| 17:58 |
|
Quest |
but why do you still prefer static vars of app scoped vars. and 2. I wonder why the comments in the stackoverflow.com question dont just say that "the static var will be just one copy" |
| 18:00 |
|
sess |
1. im not sure how app scope works in struts, but a static map is simpler and will be accessible from anywhere, not just in struts classes |
| 18:00 |
|
sess |
2. im sure they say something like that, but in a more complex way |
| 18:01 |
|
Quest |
a comment paste: Yes, that's essentially correct. Don't do it, though. Calling a static from all over the place is a pretty clear sign that your design is bad. – Ian McLaird 3 mins ago |
| 18:01 |
|
Quest |
why would you say that? whats bad in calling one static? trillion times? |
| 18:01 |
|
Quest |
sess, got your point |
| 18:01 |
|
sess |
i think his comment is not quite correct |
| 18:02 |
|
sess |
static things can be good or bad design depending on how it's used |
| 18:02 |
|
sess |
people call static things all the time fromall layers of an application |
| 18:03 |
|
Quest |
hm |
| 18:04 |
|
Quest |
hm |
| 18:06 |
|
* pdurbin |
discovers JSTL: http://stackoverflow.com/questions/6225659/display-vaues-of-key-and-value-of-hashmap-in-jsf-page/6225733#6225733 |
| 18:07 |
|
sess |
are you using jsf without knowing EL :V |
| 18:08 |
|
pdurbin |
:/ |
| 18:08 |
|
pdurbin |
:) |
| 18:08 |
|
Quest |
me no |
| 18:08 |
|
sess |
was refering to pdurbin |
| 18:09 |
|
Quest |
sess, what collection/map would you preffer. the db table has 5 columns of int/strings and about 2,000,000 rows |
| 18:09 |
|
Quest |
ConcurrentHashMap or ConcurrentHashTable |
| 18:10 |
|
sess |
Quest: how will you lookup data from it? |
| 18:10 |
|
sess |
do you want single elements by ID? |
| 18:10 |
|
sess |
or do you want groups by some criteria |
| 18:11 |
|
sess |
selection of collection is all about access method |
| 18:11 |
|
Quest |
the table is of ips. it has a start rang, end rang, country name, country shortname |
| 18:12 |
|
sess |
i dont care about the data |
| 18:12 |
|
sess |
i wonder how you will access the data |
| 18:12 |
|
sess |
do you havea primary key when you will access the map? |
| 18:12 |
|
sess |
or do you want to search for ips by data |
| 18:13 |
|
Quest |
usual queries are select contryName from IP where 'ip' is BETWEEN startRang AND endRang |
| 18:13 |
|
sess |
how many rows are returned by a typical query? |
| 18:14 |
|
Quest |
one row will be returned for that query |
| 18:14 |
|
sess |
to start with; is there even a need for a cache? is data fetched so frequently and in huge amounts? |
| 18:14 |
|
Quest |
but the query is ver very very very frequent for all users |
| 18:14 |
|
Quest |
yes^ |
| 18:14 |
|
sess |
is the data fetch triggered by users? |
| 18:15 |
|
Quest |
this table makes all the db go down. we have many users |
| 18:15 |
|
Quest |
no triggers |
| 18:15 |
|
sess |
oh |
| 18:15 |
|
Quest |
the start and end collumns are indexed though |
| 18:15 |
|
sess |
if i may ask |
| 18:15 |
|
sess |
what is the purpose of the table? |
| 18:15 |
|
sess |
it sounds weird |
| 18:15 |
|
cem_ |
Quest sess i didnt get about cache get huge table and ? |
| 18:15 |
|
Quest |
to show the country name of the user by his ip |
| 18:16 |
|
Quest |
sess, oh you mean trigger by user. yes. kind of. by the app when user logs in. |
| 18:16 |
|
sess |
but you only need it once per session |
| 18:17 |
|
Quest |
yes. but there are so many sessions. so many users |
| 18:17 |
|
sess |
so basicly you want to select a single row of data |
| 18:17 |
|
sess |
like what |
| 18:17 |
|
Quest |
yes |
| 18:17 |
|
sess |
1000 times per hour..? |
| 18:17 |
|
sess |
how many users do you have |
| 18:17 |
|
Quest |
per minut |
| 18:17 |
|
Quest |
the query rate . over all is 350/s |
| 18:18 |
|
sess |
i dont believe that |
| 18:18 |
|
Quest |
for the ful app^ |
| 18:18 |
|
sess |
are you facebook? |
| 18:18 |
|
Quest |
not just for ip table |
| 18:18 |
|
sess |
oh |
| 18:18 |
|
Quest |
its the top app in the country |
| 18:18 |
|
sess |
so you have a few thousand concurrent users |
| 18:18 |
|
Quest |
ya |
| 18:18 |
|
sess |
that shouldnt be a problem for this table |
| 18:18 |
|
sess |
with good indexes |
| 18:18 |
|
cem_ |
Quest san please tell me about map and you get all the table |
| 18:19 |
|
Quest |
the start and end collumns are indexed though |
| 18:19 |
|
sess |
that said, isnt there another way to do this |
| 18:19 |
|
sess |
than storing all ip ranges |
| 18:19 |
|
Quest |
sess, but this table is making it down. in slow queries. this query takes most time |
| 18:19 |
|
Quest |
sess, like what way? |
| 18:19 |
|
sess |
how long time to fetch once? |
| 18:20 |
|
Quest |
that i dont have right now. the stats atm. i gues more than 10 seconds or 15 |
| 18:20 |
|
sess |
lol |
| 18:20 |
|
Quest |
iam not sure though. |
| 18:20 |
|
sess |
you need to redo your indexes |
| 18:20 |
|
sess |
should be possible to get it down fraction of a second |
| 18:20 |
|
Quest |
the indexes are unique |
| 18:20 |
|
sess |
how are you storing the ip? |
| 18:20 |
|
Quest |
redo? what do you mean. |
| 18:20 |
|
Quest |
indexes are indexes. |
| 18:20 |
|
sess |
simple string? |
| 18:20 |
|
Quest |
yes |
| 18:20 |
|
Quest |
wait. no. ints |
| 18:21 |
|
sess |
4 int columns? |
| 18:21 |
|
sess |
8 even |
| 18:21 |
|
sess |
with start and end ranges |
| 18:21 |
|
Quest |
am. i gues its 6 or 8 |
| 18:21 |
|
Quest |
each. start and end. |
| 18:21 |
|
Quest |
one solution is memcached.org but we didnt tried it yet |
| 18:22 |
|
sess |
where are probably good web services for this |
| 18:22 |
|
sess |
though with that many users you might want to store them yourselves |
| 18:22 |
|
Quest |
hm |
| 18:22 |
|
sess |
how does your query look? |
| 18:22 |
|
sess |
an example |
| 18:23 |
|
Quest |
usual queries are select contryName from IP where 'ip' is BETWEEN startRang AND endRang |
| 18:23 |
|
sess |
you said IP were stored as numbers |
| 18:23 |
|
sess |
it cant look like that |
| 18:23 |
|
Quest |
IP == IpTable |
| 18:23 |
|
Quest |
yes |
| 18:24 |
|
sess |
isnt it split into octets? |
| 18:24 |
|
Quest |
numbers without .dots |
| 18:24 |
|
sess |
did you come up with this yourself? |
| 18:24 |
|
Quest |
ips are converted to numbers by some InetAddress thing |
| 18:24 |
|
Quest |
sess, its something called max.. bbla bla.. a standard way of converting ips to numbers |
| 18:25 |
|
sess |
yeah yeah but the decision of converting to a single number and putting them into the db |
| 18:25 |
|
sess |
was the data like that when you got it? |
| 18:25 |
|
sess |
or did you do that just for convenience |
| 18:25 |
|
scripty |
350/s thats WOW |
| 18:26 |
|
Quest |
sess, affirmative |
| 18:26 |
|
Quest |
it was like it |
| 18:26 |
|
Quest |
i think.. they used http://download.java.net/jdk7/archive/b123/docs/api/java/net/InetAddress.html#toString() |
| 18:27 |
|
sess |
tbh im not sure if that is optimized for db indexing |
| 18:27 |
|
Quest |
if they are correct. as far as db is concerned. seems like the best approach to me |
| 18:28 |
|
sess |
http://stackoverflow.com/questions/18721663/efficient-table-structure-or-indexing-for-searchable-ip-address-ranges-in-sql |
| 18:28 |
|
sess |
ss |
| 18:28 |
|
sess |
ee |
| 18:28 |
|
sess |
seems like he has your issue |
| 18:28 |
|
sess |
its probably the indexes that are setup wrong |
| 18:28 |
|
sess |
no accepted answer though |
| 18:28 |
|
Quest |
sess, indexes are indexes. if they are setup on start, end columns. they are good. arnt they? |
| 18:29 |
|
sess |
indexes can be complex |
| 18:30 |
|
sess |
do you have a combined index for both fields? |
| 18:30 |
|
Quest |
yes. but seems straight forward here |
| 18:30 |
|
Quest |
I set indexes for start and end column seperately |
| 18:30 |
|
sess |
i think only one of them is used then |
| 18:30 |
|
sess |
possibly |
| 18:30 |
|
sess |
try removing them and making one combined |
| 18:30 |
|
Quest |
both should be used |
| 18:30 |
|
Quest |
hm. |
| 18:31 |
|
sess |
do an EXPLAIN query |
| 18:31 |
|
sess |
and see the execution plan |
| 18:31 |
|
sess |
should be available in most sql vendors |
| 18:31 |
|
Quest |
like CREATE INDEX startEndIndex ON ipTable (star,end) |
| 18:31 |
|
sess |
ive had weird issues where a suboptimal index is used, causing performance to dip |
| 18:31 |
|
sess |
yeah something like that |
| 18:32 |
|
sess |
it will probably take ages |
| 18:32 |
|
Quest |
how would a combined index be different from 2 individuale indexes? |
| 18:32 |
|
Quest |
sess, take ages for what? |
| 18:32 |
|
sess |
only one index can be applied |
| 18:32 |
|
sess |
i think |
| 18:32 |
|
sess |
so perhaps now, only one of your 2 indexes is used |
| 18:32 |
|
sess |
ages for creating the index |
| 18:32 |
|
Quest |
hm |
| 18:33 |
|
sess |
so first try the combined index |
| 18:33 |
|
sess |
if its still slow |
| 18:33 |
|
sess |
experiment with EXPLAIN query |
| 18:33 |
|
Quest |
sess, in a query. can 5 or 6 indexes be used at the same time? hypothetical example but can be practical in large tables? |
| 18:33 |
|
sess |
it should NOT take 15s with proper indexes |
| 18:33 |
|
sess |
im no expert, but i think the cases where multiple indexes can be used is very limited |
| 18:33 |
|
cem_ |
can anyone explain quest primary question loading large table 30 min ????? |
| 18:34 |
|
sess |
cem_: what about it |
| 18:34 |
|
Quest |
sess, thx |
| 18:34 |
|
cem_ |
sess i didnt get it |
| 18:34 |
|
sess |
cem_: im not sure what you are asking though |
| 18:35 |
|
cem_ |
<Quest> the intention is to load a large table into a Map in cache/ram (a variable). update it every 30 mins. |
| 18:35 |
|
sess |
yes? |
| 18:36 |
|
cem_ |
you answered him but i didnt get it |
| 18:37 |
|
sess |
what part did you not get |
| 18:38 |
|
Quest |
cem_, read the post again. point 5 |
| 18:41 |
|
cem_ |
how will you serve a query table using a map i know this is silly(for you guys) but its a map with key and value does this means you add to list then to map with key? |
| 18:43 |
|
sess |
cem_: youre correct, the map solution does not work well with the ips |
| 18:43 |
|
sess |
i did not know he did not have single primary keys |
| 18:44 |
|
sess |
for caching IPs, id probably make a hierarchy of maps per octet or something |
| 18:44 |
|
cem_ |
sess what is wrong with multiple primary key and why does ip table needs to change ? |
| 18:44 |
|
sess |
maps only have a single key |
| 18:44 |
|
sess |
and you cant query against them |
| 18:46 |
|
cem_ |
sess since he is doing query in DB that doesnt mean he cant have multiple keys(Am i correct) |
| 18:46 |
|
sess |
in a db he can have multiple keys ofc |
| 18:46 |
|
cem_ |
but what you where pointing too? |
| 18:46 |
|
sess |
im not pointing anywhere |
| 18:47 |
|
cem_ |
[00:13] <sess> i did not know he did not have single primary keys |
| 18:47 |
|
sess |
i did know know he was handling IPs |
| 18:48 |
|
sess |
if he has single column primary keys, he could simple put them all in a map |
| 18:49 |
|
cem_ |
all those in a map ? |
| 18:50 |
|
sess |
yes |
| 18:50 |
|
sess |
as key value pairs |
| 18:50 |
|
sess |
primary key as key |
| 18:51 |
|
sess |
rest of the entity as value |
| 18:51 |
|
cem_ |
but that is quite huge |
| 18:51 |
|
sess |
doesnt matter, just get more RAM |
| 18:52 |
|
sess |
access will be instant |
| 18:55 |
|
Quest |
so no matter how many columns you want to add index to, if the query involves all of them in the where clause. the index should be ONE with (colum1,2,3,4) not index (col1), index (col2) so on |
| 18:55 |
|
cem_ |
okay |
| 18:56 |
|
sess |
Quest: i think so yeah |
| 18:56 |
|
Quest |
it was a question to sess |
| 18:56 |
|
Quest |
hm ok |
| 18:56 |
|
sess |
they cant be combined just like that |
| 18:56 |
|
scripty |
k i have informed about bugs on there website |
| 18:56 |
|
sess |
ive learned recently |
| 18:57 |
|
Quest |
scripty, you learned by * scripty giggles ing? |
| 18:57 |
|
Quest |
:) |
| 18:57 |
|
scripty |
it takes max about 15 min to make there website disappear haha |
| 18:57 |
|
scripty |
but anyway i have informed the admin |
| 18:57 |
|
Quest |
scripty, whos web? |
| 18:57 |
|
scripty |
who knows :P |
| 18:57 |
|
Quest |
scripty, and how do you know its tcm? |
| 18:58 |
|
Quest |
sess, they cant be combined? amm why. what do you mean |
| 18:59 |
|
scripty |
lol Quest its not done like that |
| 18:59 |
|
Quest |
k :) |
| 19:00 |
|
scripty |
Quest is that your website http://www.tcm.com/ |
| 19:00 |
|
Quest |
no |
| 19:00 |
|
scripty |
Oh! |
| 19:29 |
|
|
[[thufir]] joined ##javaee |
| 20:12 |
|
|
[[thufir]] joined ##javaee |
| 20:28 |
|
pdurbin |
ah... always feels good to get code safely off my laptop and into a branch on github: https://github.com/IQSS/dvn/tree/oiosaml |
| 20:43 |
|
|
tommmied joined ##javaee |
| 20:56 |
|
Quest |
indexes are also updated on each update query while it facilitates a select query for fastness. 1. is the full index is updated for a on row update? 2. is it not worth to use indexes if the update queries exceed select queries? |
| 21:29 |
|
|
[[thufir]] joined ##javaee |
| 21:32 |
|
|
tommmied left ##javaee |
| 21:47 |
|
|
[[thufir]] joined ##javaee |
| 22:14 |
|
|
[[thufir]] joined ##javaee |
| 23:17 |
|
|
MegaMatt joined ##javaee |
| 23:19 |
|
|
[[thufir]] joined ##javaee |
