# Obtaining Cyanogenmod source code without repo sync



## galapogos

Hi,

Is it possible to download the CM source code without using repo sync? I am on a network that has a firewall that seems to block access, so the only other way I can think of is to download it elsewhere and copy the whole source code afterwards.

Thanks.


----------



## yarly

repo is just a wrapper for git. It's using http (port 80) or https (port 443) to connect to github.com to get the sources. If you arent able to connect to those ports, then you wouldn't be able to post the message you left above


----------



## galapogos

I'm not sure what the firewall does, but for some reason I'm unable to run the repo commands to get the source. If I unplug the network cable and tether off my 3G phone it works. However I'll obviously bust my data plan trying to download the entire source for my device.

In any case, can you please answer my question?

Thank you.


----------



## yarly

I did. Do you want me to fly to wherever you are and deliver the source code to you in person? I suppose I could also provide it via carrier pigeon with a usb drive attached to its leg. Fix whatever is wrong with your connection, do it from somewhere other than your workplace or find a wifi hotspot, because git is not doing anything fancy. Go to github itself and try using git manually. You really haven't tested much of anything and just gave up and assume it's your "firewall."

You're also being awfully pushy for someone that wants help (I don't like people trying to demand answers from me or anyone else as this is a volunteer forum). Maybe English isn't your first language. If it isn't, I can forgive the comment about thinking you're pushy.


----------



## DarthG

galapogos said:


> I'm not sure what the firewall does, but for some reason I'm unable to run the repo commands to get the source. If I unplug the network cable and tether off my 3G phone it works. However I'll obviously bust my data plan trying to download the entire source for my device.
> 
> In any case, can you please answer my question?
> 
> Thank you.


When in doubt, go to Google.com
Also, when you're at work, do what you were paid to do... your job!


----------



## galapogos

yarly said:


> I did. Do you want me to fly to wherever you are and deliver the source code to you in person? I suppose I could also provide it via carrier pigeon with a usb drive attached to its leg. Fix whatever is wrong with your connection, do it from somewhere other than your workplace or find a wifi hotspot, because git is not doing anything fancy. Go to github itself and try using git manually. You really haven't tested much of anything and just gave up and assume it's your "firewall."
> 
> You're also being awfully pushy for someone that wants help (I don't like people trying to demand answers from me or anyone else as this is a volunteer forum). Maybe English isn't your first language. If it isn't, I can forgive the comment about thinking you're pushy.


I did not ask you, or anyone, to do any of that, so I'm not sure why you're being sarcastic. All I asked is for another method to obtain the source code - some mirror where I can download it on another computer on another network without the limitations of my current one. I didn't realize it was that difficult to provide me with a link. I do not have control of the network settings - it isn't my network, so there's nothing I can do to "fix" it.

I also didn't realize that explaining my problem, and asking for someone to please answer it directly instead of basically telling me that it should be working, constitutes as pushy behavior around here. I appreciate you educating me that repo is not doing anything funny - that's enlightening, but it doesn't solve my problem. The fact is that it's not working, hence I'm finding alternatives.


----------



## yarly

> In any case, can you please answer my question?


↑ Sounds pushy to me, since you didn't refute my reply about whether english was your primary language.

I told you to troubleshoot your issue by using github and grabbing a repository there manually using git. If you don't want to do that (so we can try to determine what the issue is) and just complain, then you will get no help from me.

Seriously if you cant do following to test to see if git will work on your connection:



Code:


git clone [URL=https://github.com/CyanogenMod/android_frameworks_base]https://github.com/CyanogenMod/android_frameworks_base[/URL].git

Then I don't think it's worth my time to help you because you just want to yell and scream and can't calm down enough to listen to someone that was trying to help you to do more than some half ass solution that would be a pain to deal with that may not even be needed. My tech help comes at a price of me finding the best solution (which can take time), not some quick hack you think will work. If you want a hack, go call some outsourced tech support hotline.


----------



## galapogos

OK, we probably got off the wrong foot. I don't set out to piss people off or appear to just want to yell and scream. In fact I was pretty calm during all my postings, but I understand that words are often prone to misinterpretation, and do not convey the proper tone that the person wishes to convey, so if I appeared pushy, that was not my intention. I believe we can both learn from the 1st link in your signature.

Back to the problem. I am, in fact, a total noob in Android source building, using repositories, or even using Ubuntu. My previous experience with Linux had been exclusively with Mandriva/Mandrake. So, I did not know what you meant by using github and grabbing a repository. I did go to github.com and tried to download the Android source code but it was split up into so many parts that I didn't know which to download.

In any case, I just ran the command that you posted, and it seems to be downloading:



Code:


<br />
Initialized empty Git repository in /home/goister/android/system/android_frameworks_base/.git/<br />
remote: Counting objects: 536823, done.<br />
remote: Compressing objects: 100% (169610/169610), done.<br />
Receiving objects:  45% (244389/536823), 176.24 MiB | 218 KiB/s  <br />

Previously when I tried a "repo sync -j8", I'd get a connection time out error after a while. Let's see if the git command works.


----------



## yarly

No worries. I just wanted you to clarify things for me so I would know what was up. Words do get misconstrued on the internet as you said.

Yeah, that was just a test (doing the git clone using github) to get the java parts of cyanogen. It's not all of the source totally. However, the fact it seems to download means it's not a problem with your connection, just one with the repo script (so the problem lies in that and has to be figured out). I can try to help you to figure out what is the issue in repo, but probably need more info.

Github is also where Cyanogen stores all their sources. You can manually clone them via http://rootzwiki.com...nc/#entry748133, but it's much better if you use the repo script if we can get it working. Otherwise it's going to make it a pain in the ass to try to update your local code when Cyanogen pushes new changes.

Also where did you get your repo script? Maybe the one you have is outdated. This is the most recent one: http://pastie.org/private/pwqo6tn9eyfxag3pitvteg


----------



## galapogos

OK, the git command successfully completed, downloading hundreds of MB of stuff.

I simply followed build instructions from http://wiki.cyanogen...ing_from_source

I'm building for the Incredible S, but since that link doesn't provide any info, I followed the instructions from the Desire GSM.

I searched for the repo script, which is in bin ~/bin/ directory. I did a diff between that and the link you provided, and there was no difference.

Anyway, I just tried to run "repo init -u git://github.com/CyanogenMod/android.git -b gingerbread" again, and I got a connection time out again:



Code:


<br />
github.com[0: 207.97.227.239]: errno=Connection timed out<br />
fatal: unable to connect a socket (Connection timed out)<br />
github.com[0: 207.97.227.239]: errno=Connection timed out<br />
fatal: unable to connect a socket (Connection timed out)<br />
fatal: cannot obtain manifest [URL=git://github.com/CyanogenMod/android]git://github.com/CyanogenMod/android[/URL].git<br />

Thanks.


----------



## yarly

Try



Code:


repo init -u [URL=https://github.com/CyanogenMod/android.git]https://github.com/CyanogenMod/android.git[/URL] -b gb-release-7.2

if that doesnt work then try



Code:


repo init -u [URL=git://github.com/CyanogenMod/android.git]git://github.com/CyanogenMod/android.git[/URL] -b gb-release-7.2

You really want that over the other anyways, since it was updated to 7.2.


----------



## galapogos

OK, the https link worked, after which I ran repo sync -j8

It's currently doing its stuff. However, the first few lines displayed are as follows:



Code:


<br />
error: '/home/goister/android/system/.repo/repo/.git/clone.bundle' does not look like a v2 bundle file<br />
fatal: Could not read bundle '/home/goister/android/system/.repo/repo/.git/clone.bundle'.<br />
Fetching projects:   1% (3/298)  remote: Counting objects: 3684, done.<br />

Is there something wrong?

Meanwhile I'll wait for it to finish doing its stuff...


----------



## yarly

Don't know. I never had an error like that. If it is, it's on your end somewhere. Was the folder you ran repo sync on empty? Otherwise, I would just ignore it for now.

You can also up the -j option in repo sync to 16 or 32 if your internet connection is fast enough. If it isnt, then it's best to keep a lower number like 4 or 8 because it'll go slower and not faster.


----------



## galapogos

OK, thanks. It's was downloading for a while, and seemed like it was downloading for all devices.

However, now it has hung on the following:



Code:


Fetching projects:  31% (93/298)

It has been there for quite a while, and there's been no network or CPU activity.


----------



## yarly

it technically does fetch for all devices because most of it is sharing the same files. that's just how it works.

I would start over and remove everything and redo repo init again and redo repo sync.

You can also try http:// with repo init instead of https. It might work better.

If it fails again, then let me know


----------



## galapogos

By "start over and remove everything" do you mean start with a clean ~/android/system directory?

I'm trying that now, with the http version.

Edit: After a few minutes, it has hung at the following:



Code:


<br />
Fetching projects:  95% (283/297)  remote: Counting objects: 1659, done.<br />
remote: Compressing objects: 100% (589/589), done.<br />
remote: Total 1659 (delta 1160), reused 1562 (delta 1064)<br />
Receiving objects: 100% (1659/1659), 5.34 MiB | 1.71 MiB/s, done.<br />
Resolving deltas: 100% (1160/1160), done.<br />
From [URL=http://github]http://github[/URL].com/CyanogenMod/android_vendor_tmobile_themes_Cyanbread<br />
* [new branch]	  gb-release-7.2 -> github/gb-release-7.2<br />
* [new branch]	  gingerbread -> github/gingerbread<br />
* [new branch]	  gingerbread-release -> github/gingerbread-release<br />
* [new tag]		 cm-7.1.0   -> cm-7.1.0<br />
remote: Counting objects: 2131, done.<br />
remote: Compressing objects: 100% (891/891), done.<br />
remote: Total 2131 (delta 1277), reused 2055 (delta 1230)<br />
Receiving objects: 100% (2131/2131), 5.89 MiB | 1.77 MiB/s, done.<br />
Resolving deltas: 100% (1277/1277), done.<br />
From [URL=http://github]http://github[/URL].com/CyanogenMod/android_vendor_tmobile_themes_Androidian<br />
* [new branch]	  gb-release-7.2 -> github/gb-release-7.2<br />
* [new branch]	  gingerbread -> github/gingerbread<br />
* [new branch]	  gingerbread-release -> github/gingerbread-release<br />
* [new branch]	  ics		-> github/ics<br />
* [new tag]		 cm-7.1.0   -> cm-7.1.0<br />











I tried killing the python script and restarting several times and it will hang at "Fetching projects" at various percentages.

I did a "repo forall -c 'echo $REPO_PATH starting ; repo sync . ; echo $REPO_PATH done'" to figure out which project it's hanging at, and I get the following output:



Code:


<br />
skipping android/<br />
skipping bionic/<br />
skipping bootable/bootloader/legacy/<br />
skipping bootable/diskinstaller/<br />
skipping bootable/recovery/<br />
skipping build/<br />
skipping dalvik/<br />
skipping development/<br />
skipping device/advent/vega/<br />
skipping device/bn/encore/<br />
skipping device/common/<br />
skipping device/commtiva/z71/<br />
skipping device/geeksphone/one/<br />
skipping device/geeksphone/zero/<br />
skipping device/htc/7x30-recovery/<br />
skipping device/htc/ace/<br />
skipping device/htc/bravo/<br />
skipping device/htc/bravoc/<br />
skipping device/htc/buzz/<br />
skipping device/htc/click/<br />
skipping device/htc/common/<br />
skipping device/htc/desirec/<br />
skipping device/htc/dream_sapphire/<br />
skipping device/htc/espresso/<br />
skipping device/htc/glacier/<br />
skipping device/htc/hero/<br />
skipping device/htc/heroc/<br />
skipping device/htc/inc/<br />
skipping device/htc/legend/<br />
skipping device/htc/leo/<br />
skipping device/htc/liberty/<br />
skipping device/htc/msm7x30-common/<br />
skipping device/htc/passion/<br />
skipping device/htc/passion-common/<br />
error: revision gb-release-7.2 in CyanogenMod/android_device_htc_saga not found<br />

Is there something wrong with my script?


----------



## yarly

Yeah a clean directory of removing anything you downloaded outside of repo.

Maybe you need to lower the -j number. Try -j4


----------



## galapogos

OK that's weird because even after the failed repo, after I killed repo yesterday, my ~/android/system directory was completely empty. Where were all the files being downloaded then?

However, today when I killed repo, there were some files in the ~/android/system directory.

I'm currently trying -j1 but that still hangs









I've also tried sudo sysctl -w net.ipv4.tcp_window_scaling=0 as recommended by some facing the same problem, but still no go.


----------



## yarly

can you list what is in that directory? If it's large, put it on pastebin.com


----------



## galapogos

Here


----------



## galapogos

Thanks for all the help btw, appreciate it.

Anyway, I'm trying a 3G dongle now and it's so far so good (though slower than ethernet), I'm keeping my fingers crossed. I'm assuming a repo sync compares what I have locally with what's on the server and syncs everything, so it works even though I had multiple failed/partial downloads earlier?


----------



## yarly

Did everything finish eventually okay?


----------



## galapogos

Sorry for being MIA.

Unfortunately, no. I get the same kinds of errors(hanging) without any error code to indicate what's going on.

I'm now toying with the idea of going to https://github.com/CyanogenMod and downloading the individual packages one by one. Not a very attractive option, but I don't really know what else to do.









I tried downloading a few using this manual method and it seems to work, but I just gotta figure out exactly which packages to download for my target device and my target version. Speaking of target version, I see several different branches. Earlier on, you suggested I use gb-release-7.2 instead of gingerbread. However, I also see cm-7.2.0 and gingerbread-release. What's the difference between all these branches?


----------



## yarly

Well you don't really want anything before 7.2, it's old and no longer maintained.

Sucks you can't get repo working. If you don't get it via using git though, you wont be able to keep things updated, so if you do get them manually, at least pull them using git like I was having you do before manually.


----------



## galapogos

Yeah it sucks indeed.









I know CM 7.2 is the latest version, but I'm wondering what the differences are between gb-release-7.2 and cm-7.2.0. Do I have to download both branches to build CM 7.2? Are they totally separate branches that are self sufficient on their own, or is one branch a subset of another? I see some packages that are in 1 branch but not the other. For example, if I were to build for the HTC Incredible S and I go to https://github.com/C...device_htc_incs there's only a gingerbread branch and not a gb-release-7.2 branch. Does this mean I need to download the gingerbread branch? The device is obviously supported by CM 7.2 as there's a device page for it - http://download.cyan...ble&device=vivo

Also, I see certain packages that are in the master branch or aosp branch, wondering if I need to download those too.

Edit: I've completed downloading all archives that are tagged in the gb-release-7.2 branch, except for android_prebuilt. I keep getting a 502 bad gateway error. Wonder if this is why repo/git fails...

Thanks!


----------



## yarly

Okay, you're going to hate me for this, but start over, lol.

I tried building CM 7.2 and you do need the normal gingerbread source tree. So try doing this (after removing everything):

repo init -u https://github.com/CyanogenMod/android.git -b gingerbread

repo sync -j16

...then wait for it to work.

For troubleshooting, look under your .repo/manifests/default.xml and make sure everything is using http or https to connect (and not git, since you have troubles with that). The .repo directory will be in the same directory you ran "repo init"


----------



## galapogos

Doh, I can't even init now. After a while it just throws this error:



Code:


<br />
fatal: unable to connect to github.com:<br />
github.com[0: 207.97.227.239]: errno=Connection timed out<br />

Edit: OK it's working, trying repo sync again this time with GIT_TRACE enabled to see what's going on...let's see how far I get...

Edit2: Not very far...getting stick at andoid_device_htc_leo with the following log output:



Code:


<br />
error: GnuTLS recv error (-9): A TLS packet with unexpected length was received. while accessing [URL=https://gerrit.googlesource]https://gerrit.googlesource[/URL].com/git-repo/info/refs<br />
fatal: HTTP request failed<br />
trace: built-in: git 'fetch' 'origin' '--tags' '+refs/heads/*:refs/remotes/origin/*'<br />
trace: run_command: 'git-remote-https' 'origin' 'https://gerrit.googlesource.com/git-repo'<br />
error: GnuTLS recv error (-9): A TLS packet with unexpected length was received. while accessing [URL=https://gerrit.googlesource]https://gerrit.googlesource[/URL].com/git-repo/info/refs<br />
fatal: HTTP request failed<br />
error: Cannot fetch repo<br />
trace: built-in: git 'fetch' 'github' '--tags' '+refs/heads/*:refs/remotes/github/*'<br />
trace: run_command: 'git-remote-http' 'github' 'http://github.com/CyanogenMod/android_device_htc_leo'<br />
trace: run_command: 'rev-list' '--objects' '--stdin' '--not' '--all' '--quiet'<br />
trace: run_command: 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'http://github.com/CyanogenMod/android_device_htc_leo/'<br />
trace: exec: 'git' 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'http://github.com/CyanogenMod/android_device_htc_leo/'<br />
trace: built-in: git 'fetch-pack' '--stateless-rpc' '--stdin' '--lock-pack' '--thin' 'http://github.com/CyanogenMod/android_device_htc_leo/'<br />

Just wondering, can I run the git commands individually based on the repo wrapper script, and if so, where do I refer to it? Adding to that, if this is possible, I can probably also just download them one by one from github.com, just that I need to download the correct branch(the script should have the full list of downloads?) and I also lose the ability to update?


----------

